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of  gunnery  conditions  and  actions.  SIMNET  can  support  training  of  much  of  the  gunnery 
domain,  despite  its  avowed  purpose  of  training  tactical  skills.  The  low-cost  gunnery 
devices  (VIGS  and  TopGun)  provide  adequate  simulations  of  the  conditions  and  actions 
related  to  precision  gunnery  from  a  stationary  tank.  Each  of  the  devices  had  one  or 
more  characteristics  that  creates  negative  transfer.  Therefore,  on-tank  experience  at 
both  the  beginning  and  advanced  stages  of  training  is  necessary  for  training  and  test¬ 
ing  gunnery. 

The  final  process  was  to  combine  the  evaluations  with  the  training  and  testing 
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1 1 


_ UNCLASSIFIED _ 

SECURITY  CLASSIFICATION  OF  THIS  PAGE 


Technical  Report  783 


Requirements  for  a  Device-Based  Training 
and  Testing  Program  for  Ml  Gunnery: 
Volume  1.  Rationale  and  Summary  of  Results 


R.  Gene  Hoffman  and  John  E.  Morrison 

Human  Resources  Research  Organization 

for 

Contracting  Officer's  Representative 

John  A.  Boldovici 


ARI  Field  Unit  at  Fort  Knox,  Kentucky 
Donald  F.  Haggard,  Chief 


Training  Research  Laboratory 
Jack  H.  Hiller,  Director 


U  S.  ARMY  RESEARCH  INSTITUTE  FOR  THE  BEHAVIORAL  AND  SOCIAL  SCIENCES 
5001  Eisenhower  Avenue,  Alexandria,  Virginia  22333-5600 


Office,  Deputy  Chief  of  Staff  for  Personnel 
Department  of  the  Army 


March  1988 


Army  Project  Number  Education  and  Training 

2Q263743A794 


Approved  for  public  release;  distribution  unlimited. 


FOREWORD 


This  report  is  the  firstof  two  volumes  describing  four  related  activities 
focusing  on  Ml  tank  crew  gunnery  performance:  (1)  an  analysis  of  the  domain  of 
tactical  gunnery,  (2)  specification  of  training  and  testing  objectives,  (3)  con 
tent  evaluation  of  four  training  devices,  and  (4)  development  of  a  training  and 
testing  strategy  using  those  training  devices. 

This  research  is  a  part  of  the  Army  Research  Institute  (ARI)  task  entitled 
"Application  of  Technology  to  Meet  Armor  Skills  Training  Needs."  That  task  is 
performed  under  the  auspices  of  ARI’s  Armor  Research  and  Development  Activity 
at  Fort  Knox,  whose  mission  includes  optimizing  the  use  of  armor  training  de¬ 
vices  for  readiness  in  gunnery  and  tactics. 

The  proponent  for  this  research  is  Training  and  Doctrine  Command  (TRADOC), 
and  the  user  is  U.S.  Army  Armor  Center  (USAARMC)  (Letter  of  Agreement  with  ARI 
entitled  "Establishment  of  Training  Technology  Field  Activity,  Fort  Knox,  Ken¬ 
tucky,"  4  November  1983). 

Plans  for,  and  progress  on,  this  project  have  been  disseminated  through 
briefings  to  the  Assistant  Commandant,  Technical  Director,  and  Department  Heads 
of  the  U.S.  Army  Armor  School  at  Fort  Knox.  Project  scientists  also  made  in¬ 
formal  presentations  to  the  Director  of  the  Armor  School  Directorate  of  Train¬ 
ing  Developments  (DOTD) ,  and  to  ORSA  (Operations  Research  and  Systems  Analysis) 
personnel.  Access  to  data  sources  was  provided  by  Gary  Elliot  of  the  Direc¬ 
torate  of  Evaluation  and  Standardization  and  by  Robert  Cisco  of  the  Office, 
Chief  of  Armor.  Additional  presentations  are  being  planned  for  DOTD  personnel 
and  the  American  Psychological  Association.  Advance  copies  of  the  report  have 
been  sent  to  PM-TRADE  and  the  TRADOC  Deputy  Chief  of  Staff  for  Training. 

The  research  provides  information  complementary  to  current  emphasis  and 
proposals  regarding  armor  device  training  strategies.  The  gunnery  training 
objectives,  summaries  of  device  capabilities  and  limitations,  and  procedures 
for  developing  device-based  training  will,  we  hope,  be  useful  at  all  levels  of 
training  and  testing  development  for 


EDGAR  M.  JOHNSON 
Technical  Director 


REQUIREMENTS  FOR  A  DEVICE-BASED  TRAINING  AND  TESTING  PROGRAM  FOR  Ml  GUNNERY: 
VOLUME  1.  RATIONALE  AND  SUMMARY  OF  RESULTS 


EXECUTIVE  SUMMARY 


Requi rement: 

Gunnery  weapons  and  training  device  technology  have  undergone  a  radical 
shift  in  capabilities  because  of  the  availability  of  small,  low-cost  computers. 
For  weapon  systems,  the  advanced  features  of  the  Ml  tank  create  greater  fight¬ 
ing  potential  but  also  create  more  extensive  crew  responsibilities.  Training 
devices  have  also  become  more  complex  with  new  stand-alone  devices  that  attempt 
to  represent  larger  segments  of  the  gunnery  domain.  In  this  report,  we  examine 
four  such  computer-based  devices  that  can  be  used  to  train  Ml  gunnery  skills: 
the  Videodisc  Interactive  Gunnery  Simulator  (VIGS),  the  arcade-type  TopGun  de¬ 
vice,  the  Unit  Conduct-of-Fi re  Trainer  (U-COFT),  and  the  Simulated  Networking 
(SIMNET)  battle  simulation  system. 

This  report  is  concerned  with  several  interrelated  questions  that  have 
arisen  because  performance  requirements  and  training  device  capabilities  are 
both  expanding.  First,  given  the  technological  sophistication  of  the  Ml  tank, 
what  is  the  domain  of  stimulus  conditions  and  behaviors  that  relate  to  tank 
gunnery?  Second,  what  are  the  training  requirements  for  tank  gunnery?  Third, 
what  are  the  capabilities  of  the  four  computer-based  devices  for  providing 
training?  Fourth,  can  an  integrated  program  be  designed  to  train  gunnery 
skills  with  those  devices? 

No  quality  training  program  can  exist  without  the  capacity  for  measuring 
students'  progress.  Therefore,  two  questions  arise  about  performance  measure¬ 
ment  capabilities  of  the  new  devices.  First,  can  devices  determine  gunnery 
performance  deficiencies?  Second,  is  performance  on  the  device  predictive  of 
performance  on  actual  equipment? 


Procedure: 

The  project  began  with  an  analysis  of  the  domain  of  gunnery  stimulus 
conditions  and  behaviors.  This  analysis  pulled  together  information  from  a 
variety  of  sources  and  organized  it  into  a  more  comprehensive  description  of 
Ml  gunnery  procedural  elements  than  were  available  from  any  one  existing 
source.  For  purposes  of  training  and  testing,  we  confined  our  attention  to 
crew  duties--one  crew  member  at  a  time — but  included  events,  conditions,  and 
actions  that  are  a  part  of  the  context  of  a  crew  operating  in  a  platoon  mis¬ 
sion.  This  added  significantly  to  the  performance  requirements  that  are  re¬ 
quired  on  a  more  "pure"  gunnery  (i.e.,  gunnery  marksmanship)  exercise  such  as 
Tank  Table  VIII.  Thus,  our  terminal  objective  for  gunnery  was  not  confined  to 
"pure"  gunnery,  but  was  expanded  to  include  gunnery  conditions  and  behaviors 
associated  with  a  tactical  context.  We  referred  to  our  domain  as  Ml  tactical 
gunnery.  Our  gunnery  domain  description  became  the  foundation  for  (a)  deriv- 
ing  training  and  testing  objectives  and  (b)  evaluating  device  capabilities. 


Our  partitioning  of  the  gunnery  domain  into  training  and  testing  objec¬ 
tives  was  based  on  a  hierarchical  analysis  approach.  According  to  this  ap¬ 
proach,  dependencies  among  performance  requirements  are  identified  and  training 
objectives  are  organized  to  ensure  that  prerequisites  are  presented  first.  To 
supplement  these  training  objectives,  we  evaluated  feedback  requirements  and 
determined  measurement  specifications  for  the  objectives. 

The  gunnery  domain  description  was  also  used  to  guide  evaluations  of  the 
four  new  gunnery  devices  for  both  training  and  testing.  Evaluations  of  device 
capabilities  were  based  on  the  fidelity  of  stimulus  and  response  representation, 
on  instructional  features,  and  on  testing  capabilities.  Of  particular  interest 
was  the  identification  of  device  features  that  might  cause  negative  transfer  of 
training  and/or  reductions  in  the  correlation  between  performance  on  the  device 
and  performance  on  the  actual  equipment  as  a  testing  activity. 

It  should  be  noted  that  these  device  evaluations  were  not  based  on  experi¬ 
mental  transfer  of  training  studies.  They  were  based  on  detailed  observation 
of  device  performance  characteristics  using  domain  description  and  instruc¬ 
tional  features  inventories. 

The  final  process  was  to  combine  the  evaluations  with  the  training  and 
testing  objectives  to  produce  a  systematic  strategy  for  training  and  testing 
gunnery  skills. 


Fi ndi ngs : 

This  volume  of  the  report  emphasizes  our  analytic  rationale  and  presents 
result  summaries  for  each  of  these  research  activities.  The  second  volume 
(Morrison  &  Hoffman,  1987)  presents  appendixes  that  describe  in  detail  the 
results  from  each  activity. 

The  tactical  gunnery  domain  was  described  by  two  lists:  (a)  a  list  of 
tank  gunnery  conditions  and  (b)  a  list  of  tank  gunnery  behaviors.  These  two 
lists  were  not  independent,  but  contained  intentional  redundancies.  That 
is,  where  conditions  created  qualitatively  different  gunnery  behaviors  (e.g., 
single  target  versus  multiple  targets),  the  behaviors  were  segregated  as  sep¬ 
arate  activities.  On  the  other  hand,  some  conditions  (e.g.,  target  cover  and 
concealment)  have  more  subtle  effects  (i.e.,  do  not  lead  to  different  behav¬ 
iors)  and  therefore  did  not  show  up  as  separate  activities. 

Gunnery  conditions  were  organized  along  the  following  22  "parameters": 

Target  type  IFFN  Formation 

Target  movement  Enemy  activity  Special  engagement 

Target  cover/  NBC  conditions  requirements 

concealment  Equipment  status  Space 

Target  array  Number  of  crewman  Visibility 

Target  orientation  Supply  shortages  Terrain  grade 

Target  range  Mission  Terrain  vegetation 

Target  sector  Fire  control  Cultural  features 


Gunnery  behaviors  were  analogously  organized  into  the  following  11 
"activities": 

Prepare  stations  for  operation 
Perform  prepare-to-fi re  checks 
Acquire  targets 

Engage  single  target  with  main  gun 
Adjust  fire 

Engage  single  target  with  the  coax 
Engage  multiple  targets  with  main  gun 

Engage  target  with  Cal  .50  (including  simultaneous  with  main  gun) 

Engage  target  using  degraded  gunnery  techniques 
Engage  main  gun  target(s)  from  the  TC  position 
Assess  results  of  engagement 

The  activities  were  further  divided  into  parts  and  options,  with  behavior  re¬ 
quirements  listed  in  detail  for  all  four  crew  members. 

The  analysis  of  training  and  testing  objectives  indicated  that,  in  gen¬ 
eral,  the  gunnery  domain  was  wide  but  not  very  deep.  By  wide,  we  meant  that 
the  student  must  know  how  to  perform  a  wide  range  of  disparate  activities.  By 
its  lack  of  depth,  we  meant  that  performance  is  not  dependent  on  a  lot  of  pre¬ 
requisite  knowledge  and  skill.  These  structural  characteristics  of  the  gunnery 
domain  had  some  general  implications  for  the  acquisition  and  retention  of  gun¬ 
nery  skills.  For  one,  initial  train-up  of  gunnery  skills  should  be  relatively 
fast  since  training  is  not  dependent  on  complex  layers  of  prerequisite  skills. 
However,  skill  sustainment  may  be  a  problem  for  two  reasons:  (1)  the  sheer 
extent  and  diversity  of  the  domain  and  (b)  the  predominance  of  procedural 
tasks,  which  research  has  indicated  are  especially  susceptible  to  decay  (e.g., 
Schendel ,  Shields,  &  Katz,  1978).  The  hierarchical  analyses  specified  only  a 
proportionally  small  number  of  required  sequences  in  instruction.  Additional 
rules  for  sequencing  instruction  were  derived  from  a  training  strategy  that  we 
termed  "progressive  elaboration"  (Bessemer,  personal  communication,  December 
1987).  According  to  this  strategy,  students  start  training  on  some  basic  pro¬ 
cedure  (e.g.,  precision  gunnery)  and  add  variations  and  elaborations  as  basic 
skills  are  mastered. 

Analysis  of  measurement  requirements  emphasized  consideration  of  the  kind 
of  feedback  that  is  most  likely  to  be  required  to  improve  deficient  perfor¬ 
mance.  Feedback  was  focused  on  any  of  three  areas:  (a)  knowledge  (i.e., 
facts  or  rules)  needed  to  conduct  the  procedure,  (b)  behavior  techniques,  and 
(c)  outcomes  of  behavior.  Specification  of  which  to  use  depends  on  decision¬ 
making  or  cognition  skills  required,  on  the  difficulty  of  the  motor  skills  re¬ 
quired,  and  how  closely  outcomes  are  tied  to  behavior.  For  example,  because 
of  the  number  of  factors  influencing  the  fall  of  a  round,  target  hits  is  not 
a  very  good  indicator  of  the  cause  of  skill  deficiencies  in  tracking. 

Evaluation  of  the  devices  was  conducted  with  respect  to  fidelity  fea¬ 
tures,  instructional  features,  and  testing  capabilities.  These  evaluations 
were  rational  assessments  of  device  capabilities  with  respect  to  tactical 
gunnery  domain  performance  elements,  instruction  requirements,  and  testing 
capacities.  Several  of  the  more  general  conclusions  from  these  analyses  are 
presented  below: 


1.  Over  the  entire  domain,  U-COFT  provides  a  comprehensive  and  realistic 
simulation  of  gunnery  conditions  and  actions.  Considering  the  purpose  and  cost 
of  U-COFT,  this  is  not  an  unexpected  result. 

2.  Despite  its  avowed  purpose  to  train  tactical  skills,  SIMNET  can  sup¬ 
port  training  of  much  of  the  gunnery  domain.  In  terms  of  gunnery  conditions, 
SIMNET  provides  a  simulation  of  the  mission  requirement  parameters  (e.g.,  fire 
control,  movement  formation)  not  supported  by  the  ether  simulators.  In  terms 
of  actions,  SIMNET  provides  a  sufficient  simulation  of  many  crew  gunnery  ac¬ 
tivities,  and  it  is  the  only  computer-based  simulator  to  provide  practice  on 
driver  and  loader  activities. 

3.  The  low-cost  gunnery  devices  (VIGS  and  TopGun)  provide  adequate  simu¬ 
lations  of  the  conditions  and  actions  related  to  precision  gunnery  from  a  sta¬ 
tionary  tank.  While  this  activity  is  but  a  small  part  of  the  gunnery  domain, 
precision  gunnery  skills  are  regarded  as  prerequisite  to  many  of  the  other  as¬ 
pects  of  gunnery. 

4.  None  of  the  devices  provides  worthwhile  instruction  or  practice  of 
secondary  gunnery  functions  (prepare  for  operations  and  prepare  for  fire  pro¬ 
cedures).  Simulation  of  the  target  acquisition  and  identification  phases  of 
gunnery  is  also  deficient  for  all  four  devices. 

5.  U-COFT  and  VIGS  exercises  start  with  some  sort  of  preview  that  in¬ 
cludes  a  description  of  the  to-be-engaged  target(s)  and  the  conditions  under 
which  the  engagement  is  to  be  conducted.  These  previews  encourage  the  student 
to  preset  switches  on  the  Fire  Control  Panel.  Not  only  is  this  inappropriate 
practice,  but  it  may  also  lead  to  the  unsafe  habit  of  arming  the  weapons  and 
the  laser  range  finder  before  the  TC  issues  a  fire  command. 

6.  Effective  use  of  devices  requires  that  they  allow  for  the  selection 
of  exercises.  All  devices  have  some  capabilities  in  this  regard,  but  SIMNET 
appears  more  cumbersome  to  use. 

7.  U-COFT  and  TopGun  appear  superior  to  both  SIMNET  and  VIGS  in  allowing 
an  instructor  to  control  (start,  stop,  freeze,  replay)  practice  engagements. 

8.  Except  for  U-COFT,  automated  scoring  features  on  the  devices  are 
limited  in  number  and  tend  to  be  outcome  oriented.  Outcome  measures  are  gen¬ 
erally  the  least  useful  for  diagnosing  performance  deficiencies.  Using  any  of 
the  devices  to  determine  the  causes  of  performance  deficiency  would  require 
the  close  involvement  of  an  instructor. 

9.  On-tank  experience  at  both  the  beginning  and  advanced  stages  of  train¬ 
ing  is  necessary  to  train  and  test  the  entire  domain  of  gunnery. 


Utilization  of  Findings: 

The  research  provides  information  that  should  be  useful  to  the  Armor 
training  community.  Of  particular  interest  are  the  gunnery  training  objec¬ 
tives  and  the  summary  of  device  capabilities  and  limitations.  In  addition, 
the  research  provides  a  model  for  the  development  of  device-based  training 
that  should  apply  to  other  Army  training  situations. 
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REQUIREMENTS  FOR  A  DEVICE-BASED  TRAINING  AND  TESTING  PROGRAM  FOR  Ml  GUNNERY: 
VOLUME  1.  RATIONALE  AND  SUMMARY  OF  RESULTS 


CHAPTER  1 
INTRODUCTION 


The  Army  is  in  a  constant  struggle  to  maximize  tank  crew  proficiency 
while  responsibly  managing  training  costs.  As  a  result,  the  past  15  years 
have  seen  a  proliferation  of  tank  gunnery  training  devices.  These  devices 
are  designed  to  significantly  increase  gunnery  practice  by  supplementing 
costly  live-fire  practice.  Fingerman,  Wheaton,  and  Boycan  (1979)  identi¬ 
fied  21  such  devices  for  training  skills  relevant  to  the  M60A1.  The 
majority  of  devices  were  tank-appended,  i.e.,  attached  to  the  tank  itself. 
Tank-appended  devices  provide  an  inexpensive  alternative  to  live  fire  by 
indicating  a  gunner's  aiming  point  at  the  time  he  squeezes  the  trigger. 

For  example,  the  M55  Laser  projects  a  visible  beam  on  a  target  board.  The 
Telfare  device  is  a  caliber  .50  machine  gun,  mounted  in  parallel  with  the 
main  gun  and  fired  at  scaled  range  targets.  Other  devices  were  stand¬ 
alone  devices  that  represent  selected  tank  displays  and  controls, 
providing  practice  for  various  aspects  of  gunnery  (e.g  the  Wiley  Burst-on- 
Target  trainer).  Like  the  tank-appended  devices,  their  emphasis  is  on 
providing  practice  and  feedback  in  tracking  and  aiming  the  main  gun  with 
the  power  control  handles. 

Both  gunnery  weapons'  technology  and  training  device  technology  have 
been  constantly  evolving  with  the  1980s,  witnessing  a  radical  shift  in 
capabilities  because  of  the  availability  of  relatively  low-cost  computers. 
For  weapon  systems,  the  advanced  features  of  the  Ml  tank  create  greater 
capabilities  and  also  more  extensive  crew  responsibilities.  For  instance, 
the  digital  ballistic  computer  instantaneously  corrects  for  cant,  wind, 
and  tube  wear  sources  of  error.  This  increase  in  weapon  capability  also 
increases  the  crew's  responsibility  to  know  how  to  detect  failures  in  the 
ballistic  computer  and  what  to  do  in  case  of  such  failures.  Training 
devices  also  have  become  more  complex  with  new  stand-alone  devices  that 
attempt  to  represent  larger  segments  of  the  gunnery  domain.  In  the 
present  report,  we  examine  four  such  computer-based  devices  that  can  be 
used  to  train  Ml  gunnery  skills:  the  Videodisc  Interactive  Gunnery 
Simulator  (VIGS),  the  arcade-type  TopGun  device,  the  Unit  Conduct-of-Fire 
Trainer  (U-COFT),  and  the  Simulated  Networking  (SIMNET)  battle  simulation 
system. 

This  report  is  concerned  with  several  interrelated  questions  that 
have  arisen  because  performance  requirements  and  training  device  capabili¬ 
ties  are  both  expanding.  First,  given  the  technological  sophistication  of 
the  Ml  tank,  what  is  the  domain  of  stimulus  conditions  and  behaviors  that 
relate  to  tank  gunnery?  Second,  what  are  the  training  requirements  for 
tank  gunnery?  Third,  what  are  the  capabilities  of  existing  devices  to 
provide  training?  Fourth,  can  an  integrated  program  be  designed  to  train 
gunnery  skills  with  those  devices? 


No  quality  gunnery  training  program  can  exist  without  capabilities 
for  measuring  the  progress  of  students.  A  fundamental  purpose  of  the 
training  devices  is  to  provide  feedback  concerning  main  gun  control. 
Therefore,  two  questions  arise  about  performance  measurement  capabilities 
of  the  new  devices.  First,  can  these  devices  identify  actual  gunnery 
performance  deficiencies.  Second,  is  performance  on  these  devices 
predictive  of  performance  on  the  Ml  tank? 

Our  plan  for  addressing  these  questions  is  represented  in  Figure  1-1 
by  a  flow  diagram  interrelating  four  research  activities. 

The  project  began  with  an  analysis  of  the  domain  of  gunnery  stimulus 
conditions  and  behaviors.  This  analysis  is  presented  in  Chapter  2.  This 
analysis  pulled  together  information  from  a  variety  of  sources  and 
organized  it  into  a  more  comprehensive  description  of  Ml  gunnery  pro¬ 
cedural  elements  than  available  from  any  one  existing  source. 

Our  gunnery  domain  description  became  the  foundation  for  (1)  deriving 
training  and  testing  objectives  and  (2)  evaluating  device  capabilities. 

An  initial  activity  used  in  deriving  training  and  testing  objectives  was 
the  identification  and  review  of  information  from  which  we  could  discern 
specific  areas  of  gunnery  that  may  be  particularly  difficult  to  perform. 
These  areas  could  then  be  given  special  attention  in  training  development. 
Chapter  3  describes  our  analysis  of  three  sets  of  information  that  were 
available  and  our  conclusions  regarding  gunnery  performance  deficiencies. 

Our  partitioning  of  the  gunnery  domain  into  training  and  testing 
objectives  was  based  on  a  hierarchical  analysis  approach.  This  approach 
looks  for  dependencies  among  performance  requirements  and  organizes 
training  to  insure  that  prerequisite  objectives  are  presented  first. 
Chapter  4  begins  with  a  description  of  this  approach  and  illustrates  its 
application  to  the  Ml  gunnery  domain.  Chapter  4  continues  with  presenta¬ 
tion  of  considerations  related  to  testing  the  knowledge  and  skills  of  the 
gunnery  domain.  This  section  emphasizes  the  need  for  more  than  a 
superficial  analysis  of  performance  measurement  requirements.  Example 
measurement  specifications  are  presented. 

The  gunnery  domain  description  was  also  used  to  guide  evaluations  of 
the  four  new  gunnery  devices  for  both  training  and  testing.  Evaluations 
of  device  capabilities  were  based  on  the  fidelity  of  stimulus  and  response 
representation,  on  instructional  features  and  on  testing  capabilities 
relevant  to  the  gunnery  domain.  Chapter  4  describes  fidelity  and  instruc¬ 
tional  features  criteria  and  then  presents  summary  evaluations  of  the 
devices  for  training.  Chapter  5  presents  testing  requirements  and  summary 
evaluations  of  the  devices  for  testing  appl ications.  Of  particular 
interest  was  the  identification  of  potential  sources  of  negative  transfer 
for  training  and  negative  correlations  between  performance  on  the  device 
and  performance  on  the  actual  equipment  as  a  testing  activity. 
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Figure  1-1.  Flow  of  research  activities  in  the  present  project 


The  final  process  was  to  combine  the  evaluations  with  the  training 
and  testing  objectives  to  produce  a  systematic  strategy  for  training  and 
testing  armor  skills.  Chapter  7  presents  a  concept  strategy  for  training 
and  testing  gunnery  skills. 

In  execution,  the  activities  described  above  were  not  discrete,  as 
issues  raised  in  "later"  steps  created  new  perspectives  for  "earlier" 
steps.  For  example,  revisions  and  reorganizations  of  the  gunnery  perform¬ 
ance  domain  were  on-going  throughout  the  effort.  This  willingness  to  go 
back  and  forth  between  phases  of  the  effort  was  limited  by  one  important 
principle:  Training  and  measurement  needs,  not  device  capabilities, 
should  drive  training  design.  Thus,  reorganizations  of  the  gunnery  domain 
at  various  stages  during  the  research  were  based  on  increased  understand¬ 
ing  of  the  relationships  among  gunnery  behaviors  and  were  not  dictated  by 
device  presentation  capabilities  or  limitations. 

This  volume  of  the  report  emphasizes  our  analytic  rationale  and 
summary  result  summaries  for  each  of  these  research  activities.  The 
second  volume  (Morrison  &  Hoffman,  1987)  presents  appendixes  that  describe 
in  detail  the  results  from  each  activity. 
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CHAPTER  2 


GUNNERY  TASK  PERFORMANCE  DOMAIN 


The  initial  requirement  of  developing  and  evaluating  any  training 
strategy  is  a  complete  description  of  the  performance  domain.  Such  a 
description  becomes  a  check  list  for  determining  coverage  of  training 
presentations  and  practice  exercises.  When  designing  instruction  using 
actual  equipment,  the  equipment  itself  provides  a  built-in  safeguard. 
Incorrect  or  incomplete  instruction  is  immediately  exposed  by  incompatible 
machine  demands.  On  the  other  hand,  when  training  is  to  be  conducted  on 
simulators,  much  greater  care  is  needed  in  describing  the  performance 
domain.  Incomplete  or  ambiguous  domain  specification  can  lead  to  mistakes 
in  device  design  that  are  discovered  only  after  negative  transfer  of 
training  effective  come  to  light.  In  order  for  this  project  to  reach  the 
twin  goals  of  (1)  evaluating  Ml  gunnery  training  devices,  and  (2)  design¬ 
ing  training  sequences  utilizing  those  devices,  a  complete  definition  of 
the  gunnery  domain  was  needed.  In  our  review  of  Army  doctrine  and 
research  materials,  we  found  no  one  source  that  pulled  together  all  of  the 
gunnery  performance  information.  Therefore,  we  decided  to  assemble  in  one 
document  an  Ml  gunnery  domain  description.  The  resulting  effort  was  a 
three-fold  analysis  of  (1)  the  major  activities  typically  associated  with 
tank  gunnery  (e.g.,  fire  main  gun,  fire  coax,  multiple  targets,  subsequent 
fire  commands,  etc.),  (2)  the  detailed  performance  elements  required  for 
each  activity  (e.g.,  turn  on  main  gun  switch,  close  ammo  doors,  announce 
"cease  fire"),  and  (3)  the  conditions  in  which  the  activities  are 
performed  (types  of  targets,  target  motion,  etc.).  As  a  result  the 
gunnery  performance  domain  was  defined  as  an  inventory  of  every  gunnery 
performance  element  with  the  elements  partitioned  into  major  activities. 
This  inventory  was  augmented  by  a  list  of  conditions  in  which  gunnery  is 
performed.  The  process  and  resulting  domain  are  described  in  the 
remainder  of  this  chapter. 


Background  Materials 

Our  analysis  of  the  Ml  gunnery  domain  began  with  FM  17-12-1,  Tank 
Combat  Tables  (1986).  This  field  manual  describes  doctrine  for  the  major 
activities  of  tank  gunnery  including  target  acquisition,  platoon  fire 
distribution  and  control,  as  well  as  individual  tank  crew  duties  during 
main  gun  and  machine  gun  engagements.  In  addition,  it  includes  a  series 
of  tank  gunnery  exercises,  Tank  Gunnery  Tables.  Tank  Tables  I  to  VII 
present  individual  exercises  leading  up  to  main  gun  firing  at  a  variety  of 
targets  from  a  single  moving  tank  in  Table  VIII.  Tank  Gunnery  Tables  IX 
to  XII  introduce  coordination  with  one  other  tank  in  a  section  and  three 
other  tanks  in  a  platoon.  Table  2-1  summarizes  these  gunnery  exercises. 

The  Tank  Gunnery  Tables  are  supplemented  in  FM  17-12-1  with  a  series 
of  Tactical  Tables  that  require  crews  to  practice  special  techniques  that 
introduce  skills  beyond  the  physical  limits  of  the  ranges  used  for  the 
Gunnery  Tables.  These  tables  are  also  arranged  for  one  tank,  two  tanks  in 


Table  2-1 


Summary  of  Ml  Gunnery  Tables 


Table  Exercise  Level 


I 

Basic  Gunnery  Skills 

Individual 

II 

Basic  Gunnery  Skills 

Individual/Crew 

III 

Basic  Training  Course 

Crew 

IV 

Basic  Qualification  Course3 

Crew 

V 

Machine  Gun  Training 

Crew 

VI 

Main  Gun  Calibration  and  Preliminary  Training 

Crew 

VII 

Intermediate  Training  Course 

Crew 

VIII 

Intermediate  Qualification  Course 

Crew 

IX 

Advanced  Training  Course 

Section 

X 

Advanced  Qualification  Course 

Section 

XI 

Advanced  Training  Course 

Platoon 

XII 

Advanced  Qualification  Course 

Platoon 

Notes.  Tables  IV,  VIII,  X,  and  XII  are  "gates"  in  that  soldiers  must 
qualify  on  them  to  be  eligible  for  subsequent  tests.  Gun  calibration 
(Table  VI)  must  be  performed  prior  to  main  gun  Tables  VIII,  X,  and  XII. 
Also,  training  courses  (Tables  IX  and  XI)  must  be  completed  prior  to 
attempting  qualification  on  X  and  XII,  respectively. 

STRAC  (FM  17-12-1,  1986)  guidelines  allocate  the  follow  numbers  of 
main  gun  rounds  per  crew  per  gunnery  table.  These  are  multiplied  by  the 
number  of  firings  per  gunnery  year  to  provide  the  annual  totals: 

3  x  Table  VI  screening  test:  9  rds  SABOT  (TPDS-T) 


1  x  Table  VI 

(excluding  screening  test): 
1  x  Table  VII: 

3  x  Table  VIII: 

Annual  Totals: 


10  rds  SABOT 
9  rds  SABOT 
48  rds  SABOT 
76  rds  SABOT 


4  rds  HEAT-TP-T 
6  rds  HEAT 
18  rds  HEAT 

28  rds  HEAT  =  104  total 


a  section,  and  four  tanks  in  a  platoon.  Tactical  Tables  A,  B,  D,  E,  G, 
and  H  are  various  movement,  formation,  action,  and  contact  drills  for 
individuals,  crews,  sections,  and  platoons.  Although  gunnery  is  not 
included  in  these  exercises,  they  illustrate  the  context  in  which  gunnery 
engagements  will  occur,  and  thus  represent  important  conditions  for 
gunnery.  In  contrast,  Tactical  Tables  C,  F,  and  I  do  involve  gunnery 
engagements  but  cannot  be  conducted  under  the  live-fire  range  conditions 
of  the  Tank  Gunnery  Tables.  Instead,  these  exercises  are  conducted  with 
MILES  gunnery  simulation  equipment.  These  tactical  engagement  exercises 
are  briefly  summarized  in  Table  2-2. 

Given  the  breadth  of  FM  17-12-1,  coordination  among  tanks  must  be 
included  as  part  of  the  gunnery  domain.  For  purposes  of  training  and 
testing,  we  want  to  confine  our  attention  to  include  crew  duties,  one  crew 
member  at  a  time,  but  to  include  events,  conditions,  and  actions  that  are 
a  part  of  that  crew  functioning  in  the  context  of  a  platoon  mission.  This 
introduces  the  need  to  attend  to  mission  requirements,  tank  sector 
assignments,  platoon  leader  fire  commands,  TC  reporting,  and  a  tank's 
position  and  movement  relative  to  its  companion  tanks  while  engaging  and 
being  engaged.  This  adds  significantly  to  the  performance  requirements 
that  are  required  on  a  more  "pure"  gunnery  exercise  like  Table  VIII. 

Thus,  our  terminal  objective  for  gunnery  is  not  confined  to  "pure"  gunnery 
(or  marksmanship  as  defined  by  Wheaton,  Fingerman,  and  Boycan,  1978),  but 
is  expanded  to  be  gunnery  in  a  tactical  context.  To  emphasize  this  point, 
we  will  refer  to  the  domain  as  Ml  tactical  gunnery.  Some  of  the  terms 
that  we  have  adopted  to  describe  the  domain  of  tactical  gunnery  are 
defined  below.  Included  are  the  primary  literature  sources  we  used  for 
each  aspect  of  the  domain. 

Primary  Functions.  Tank  gunnery  is  usually  described  categorically, 
differentiating  a  number  of  topics.  These  include  main  gun  single, 
multiple  and  simultaneous  targets,  coax  targets,  fire  adjustment  and 
subsequent  fire  commands.  The  principle  literature  source  for  these 
primary  functions  was  the  Tank  Combat  Tables  (FM  17-12-1). 

Secondary  Functions.  Tank  gunnery  performance  is  always  preceded  by 
gunnery  preparations  including  prepare  for  operations  (PREOPS)  procedures 
and  prepare  to  fire  (PREFIRE)  checks.  Many  of  these  procedures  give 
information  needed  for  gunnery  (e.g.  operational  status  of  the  tank). 
Others  are  important  influences  on  tank  performance  (boresighting). 

Because  of  their  necessity  for  performance,  these  were  included  in  our 
definition  of  the  tank  gunnery  domain.  The  principle  source  for  secondary 
functions  was  the  Ml  Operators  Manual  (TM  9-2350-255-10-2). 


Degraded  Mode  Techniques.  The  technological  sophistication  of  the  Ml 
creates  an  extensive  set  of  possible  system  failures  and  degraded  mode 
techniques  for  adapting  to  these  failures.  The  array  of  degraded  mode 
techniques  was  included  in  our  gunnery  domain.  Sources  of  these  condi¬ 
tions  included  FM  17-12-1,  the  Ml  Degraded  Mode  Gunnery:  Booklets  2  and  j 
(Kraemer,  1984)  and  the  degraded  mode  gunnery  courseware  for  the  hand¬ 
held  tutor  prototype  developed  for  ARI  by  Educational  Testing  Service. 
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C.  Crew  Reaction  Exercises 

1.  Engage  targets  simultaneously  with  all  three  machine  guns. 

2.  Engage  helicopter. 

3.  Engage  tanks  to  rear. 

4.  Engage  surprise  target  at  close  range. 

5.  React  to  ambush. 

6.  Engage  sniper  with  dismounted  loader  (tank  stopped  at 
obstacle) . 

F.  Section  Reaction  Exercises 

1.  Engage  multiple  targets  with  machine  guns  and  main  gun. 

2.  Engage  multiple  machine  gun  targets. 

3.  React  to  ambush. 

4.  React  to  OPFQR  tank  platoon. 

5.  Engage  patrol  and  sapper  (tanks  stopped  by  mine  field). 

6.  Engage  wheel  vehicles  and  helicopter 

I.  Platoon  Reaction  Exercises 

1.  Multiple  machine  gun  targets. 

2.  Engage  targets  of  opportunity  (reqimental  CP). 

3.  Engage  aerial  targets  (helicopter). 

4.  Engage  supply  convey. 

5.  React  to  ambush. 

6.  Engage  targets  of  opportunity  (artillery). 

7.  Engage  targets  from  hasty  battle  position. 


Conditions .  Certainly,  there  are  environmental  conditions  that  to 
varying  degrees  influence  gunnery.  Delineation  of  these  conditions  must 
be  included  in  any  specification  of  the  gunnery  domain.  Considerable 
effort  had  been  devoted  to  defining  the  domain  of  gunnery  behaviors  for 
the  M60  series  tank  based  on  identification  of  gunnery  conditions. 

Kraemer,  Boldovici,  and  Boycan  (1975),  updated  by  Wheaton,  Fingerman,  and 
Boycan  (1978),  identified  specific  gunnery  objectives  based  on  combina¬ 
tions  of  eleven  conditions.  Kraemer  et  al.  (1975)  identified  225 
objectives;  Wheaton,  et  al.  (1978)  identified  266.  These  objectives  were 
then  clustered  into  similar  types  by  examining  the  behavioral  elements 
they  had  in  common.  For  Wheaton,  Fingerman,  and  Boycan  (1978),  the  266 
objectives  were  clustered  into  16  types  of  engagements.  This  research 
represents  a  valuable  resource,  but  for  two  reasons  the  work  could  not 
simply  be  extended  to  include  Ml  gunnery.  First,  Wheaton,  et  al.  (1978) 
confined  their  analysis  to  tank  “marksmanship"  and  therefore  excluded 
tactical  context  requirements.  Second,  because  of  significant  equipment 
differences  between  the  M60  and  Ml  tanks,  seven  of  the  sixteen  M60 
clusters  do  not  apply  to  the  Ml.  Four  more  clusters  do  not  apply  because 
of  the  elimination  of  HEP  and  BEEHIVE  ammunition  from  the  basic  ammunition 
load. 


Gunnery  Domain.  Our  resulting  definition  of  the  tactical  gunnery 
domain  is  described  by  two  lists:  (a)  a  list  of  tank  gunnery  behaviors, 
and  (b)  a  list  of  tank  gunnery  conditions.  These  two  lists  are  not 
independent,  but  contain  intentional  redundancies.  That  is,  where 
conditions  create  qualitatively  different  gunnery  behaviors  (e.g.  single 
target  versus  multiple  targets),  the  behaviors  are  segregated  as  separate 
activities.  On  the  other  hand,  some  conditions  (e.g.  target  cover  and 
concealment)  have  more  subtle  effects  (i.e.,  do  not  lead  to  different 
behaviors)  and  therefore  do  not  show  up  as  separate  activities.  For 
example,  there  is  little  difference  between  descriptions  of  procedures  for 
engaging  moving  versus  stationary  targets.  However,  there  are  motor 
differences  that  make  the  condition  important  to  recognize  in  training  and 
testing.  The  following  sections  describe  the  two  components  of  the 
gunnery  domain  in  detail. 


Tactical  Gunnery  Conditions 

As  noted  above  there  are  conditions  that  are  inherent  in  the  delinea¬ 
tion  of  the  primary  functions  of  gunnery  and  there  are  other  conditions 
with  more  subtle  influence.  Using  research  literature  sources  and  in- 
house  and  ARI  gunnery  expertise,  our  approach  was  to  brainstorm  as  many 
conditions  as  possible.  Then  as  the  organization  of  elements  into 
activities  proceeded,  these  conditions  could  be  identified  as  those  that 
were  major  influences  on  gunnery  in  that  they  create  or  alter  behavior 
requirements  and  those  that  may  affect  performance  in  less  obvious  ways. 
For  example,  type  of  target  being  engaged  determines  weapon  used  and 
therefore  creates  a  significant  alteration  in  gunnery  behaviors  required. 
On  the  other  hand,  terrain  vegetation  may  influence  how  well  targets  can 
be  detected,  but  does  not  otherwise  change  the  behavior  requirements  for 
searching. 


The  results  of  our  brainstorming  of  tactical  gunnery  conditions  are 
described  in  Table  2-3.  Twenty-two  different  environmental  parameters  are 
identified  along  with  the  conditions  defining  each  parameter.  They  were 
compiled  from  a  variety  of  sources  including  FM  17-12-1  (1986)  and  Kraemer 
(1983)  for  Ml  specific  guidelines.  In  addition,  M60-series  tank  engage¬ 
ment  conditions  as  described  by  Kraemer,  Boldovici,  and  Boycan  (1975); 
Melching,  Campbell,  and  Hoffman  (1982);  and  Wheaton,  Fingerman,  and  Boycan 
(1978)  were  also  reviewed.  Finally,  platoon  tactics  doctrine,  as 
described  in  FC  17-15  (Division  86  Tank  Platoon),  was  reviewed  to  insure 
inclusion  of  platoon  level  context. 

Referring  to  Table  2-3,  Parameters  1  through  9  are  threat  parameters. 
Together  with  Parameter  10  (NBC  Conditions),  they  are  sufficient  to 
describe  most  threat  situations  that  may  be  confronted.  Parameters  11, 

12,  and  13  give  own  vehicle,  personnel,  and  supplies  status.  Parameters 
14  through  18  describe  mission  requirements.  In  addition  to  distinguish¬ 
ing  between  offense  and  defense,  these  parameters  introduce  conditions 
related  to  fire  distribution  and  control,  and  movement  and  coordination 
among  tanks  in  a  section  or  in  a  platoon.  These  are  graded  aspects  of 
"tactical  proficiency"  in  Gunnery  Tables  IX  through  XII  where  points  are 
awarded  for  fire  commands,  fire  distribution  and  control,  tactical 
movement,  and  tactical  reporting.  Specific  techniques  (e.g.  frontal 
versus  cross  fire)  are  described  in  FM  17-12-1  and  FC  17-15.  Parameter  17 
(Special  Engagement  Techniques)  includes  the  condition  that  contact  is  to 
be  avoided  (by-pass).  Finally,  parameters  19  through  22  describe  environ¬ 
mental  conditions  that  affect  gunnery  outcomes. 

Again,  many  of  these  conditions  determine  appropriate  behavior  and 
are  apparent  in  the  organization  of  gunnery  activities  below,  while  others 
may  have  more  subtle  effects  including  difficulty  of  the  activities  (e.g. 
zig-zag  moving  versus  stationary  targets)  and  influence  on  crews'  affec¬ 
tive  and  motivational  dispositions  underlying  their  gunnery  performance 
(e.g.,  low  ammunition  supplies).  This  second  type  of  condition  does  not 
require  crews  to  substantially  change  the  behaviors  as  laid  out  in  the 
activities  analysis.  Rather,  they  represent  conditions  that  make  the 
activities  more  difficult  or  distractions  that  can  disrupt  that  behavior 
and,  conse-quently,  reduce  performance.  For  example,  target  concealment 
and  NBC  conditions  can  increase  performance  difficulty  and  perceived 
threat,  but  do  not  create  qualitatively  different  performance  require¬ 
ments.  For  convenience  in  later  discussion,  we  will  refer  to  the  those 
conditions  that  are  directly  incorporated  into  the  organization  of  gunnery 
activities  as  primary  conditions,  and  those  effects  that  are  not  so 
obvious  as  secondary  conditions.  The  comments  in  Table  2-3  identify  the 
conditions  as  primary  or  secondary. 

FM  17-12-1  defines  five  "basic  factors"  of  gunnery.  These  include 
(1)  firing  tank  motion,  (2)  target  motion  and  number  of  targets, 

(3)  visibility,  (4)  sight  used,  primary  or  auxiliary,  and  (5)  engagement 
technique,  battlesight  or  precision.  Only  one  of  these,  visibility,  is 
included  as  such  in  our  analysis.  Firing  tank  motion  is  incorporated  in 
parameters  14  (Mission)  and  17  (Special  engagement  techniques).  The 
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Table  2-3 


Ml  Tactical  Gunnery  Conditions 


Parameters  Conditions  Comments 


1.  Target  type  Main  Gun 

a.  Tank 

b.  Personnel  carrier 

c.  Helicopter 

d.  Bunker 
Machinegun 

e.  Antitank 

f.  Truck 

g.  Troops  (including  antitank 
grenade  launchers  and 
antitank  grenade  missile 
teams) 

h.  Fixed  wing,  high 
performance  aircraft 

2.  Target  Stationary 

movement  a.  Front 

b.  Flank 

c.  Oblique 
Moving 

d.  Flank 

e.  Oblique 

f.  Zig-zag 

g.  Approaching 

h.  Retreating 


Distinction  between  main 
gun  and  machinegun 
targets  is  primary. 
Within  main  and 
machinegun  conditions, 
targets  are  secondary. 


Stationary  versus  moving 
is  primary  in  that  it 
affects  the  need  to 
track;  otherwise  the 
conditions  are  secondary. 


3.  Target 
cover/ 
concealment 


a.  Fully  exposed 

b.  Hull  defilade 

c.  Turret  defilade 

d.  Fully  hidden 


Secondary  conditions 


4.  Target  array  a.  Single  targets  in  distinctly  Primary  conditions 

separate  engagements 

b.  Multiple  targets  in 
distinctly  separate 
engagements 

c.  Continuous,  unpredictably 
appearing  targets  such  that 
some  appear  individually  and 
some  appear  with  others 


(table  continues) 
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Parameters  Conditions 


Comments 


5.  Target 

a.  Threat  weapons  oriented  on 

Secondary  conditions 

orientation 

own  tank 

(affects  how  targets  are 

b.  Threat  weapons  oriented 

classified,  but  doesn't 

elsewhere 

change  activities) 

6.  Target  range 

a.  Up  to  900  (Coax  tracer  burn¬ 

These  range  intervals 

out) 

(along  with  target  type) 

b.  900-1800  (Cal  .50  maximum 

are  primary  conditions 

effective  range) 

for  weapon  selection. 

c.  1800  and  over 

Additional  intervals 
within  each  category  are 
secondary  conditions  for 
range  estimation  and 
evaluation  of  laser 
returns.  Additional 
range  intervals,  base  on 
the  error  tolerance  of 
the  ballistics  system 
(e.g.  +/-200  meters) 
should  be  considered  part 
of  the  gunnery  domain. 

7.  Target  sector 

a.  Forward 

b.  Flanks 

c.  Rear 

Secondary  conditions 

8.  IFFN 

a.  All  threat 

Primary  conditions 

( identify 

b.  All  friendly 

friend  or  foe 
nomenclature) 

c.  Mix 

9.  Enemy 

a.  No  contact 

Secondary  conditions 

activity 

b.  Direct  fire 

c.  Indirect  fire 

d.  Obstacles 

e.  Minefields 

f.  Electronic  countermeasures 

10.  NBC 

a.  Free  of  hazards 

Secondary  conditions 

(nuclear , 

b.  M0PP  Level  1 

biological , 

c.  M0PP  Level  2 

chemical ) 

d.  M0PP  Level  3 

conditions 

e.  M0PP  Level  4 

(table  continues) 


Parameters 


Conditions 


Comments 


11.  Equipment  a.  Fully  operational  Secondary  conditions 

status  b.  GPS  failure 

c.  Lead  angle  failure 

d.  TIS  failure  (night) 

e.  Crosswind  sensor  failure 

f.  Cant  sensor  failure 

g.  Loss  of  symbology 

h.  Stabilization  failure 

i.  LRF  failure 

j.  Turret  power  failure 


k.  Computer  failure 

12.  Number  of 

a.  Four 

Primary  conditions 

crewman 

b.  Three 

13.  Supply 

a.  Ammo 

Primary  with  respect  to 

shortages 

b.  Fuel 

ammon  selection, 

c.  Food 

otherwise  conditions  are 

d.  Smoke  grenades 

secondary. 

e.  Other 

f.  None 


14.  Mission  a.  Moving  offense  Primary  conditions 

b.  Stationary  defense 

15.  Fire  control  a.  Single  tank  Secondary  conditions 

Section  Control  (affects  target 

b.  Frontal  selection) 

c.  Cross 

d.  Depth 
Platoon  Control 

e.  Frontal 

f.  Cross 

g.  Depth 


16.  Formation  a.  Column  Secondary  conditions 

b.  Echelon  left/right 

c.  Staggered  column 

d.  Line 

e.  Wedge 

f.  Herringbone 

g.  Vee 

h.  Coil 


Parameters 


Conditions 


Comments 


17.  Special 
engagement 
requirements 

a.  Surprise  targets 

b.  Assault  fire 

c.  Support  by  fire 

d.  Fire  and  maneuver 

e.  By-pass 

Secondary  conditions 

18.  Space 

Offensive 

a.  Support  by  fire  position 
interval 

b.  Fire  and  maneuver  interval 

c.  Assault  interval 

Defense 

d.  Fire  position  interval 

Secondary  conditions 

19.  Visibility 

Day 

a.  Unlimited 

b.  Haze,  smoke,  rain,  snow  or 
fog 

Night 

c.  Without  illumination 

d.  With  continuous  illumination 
(e.g.,  fires,  moon) 

e.  With  periodic  illumination 
(e.g.,  flares) 

Primary  with  respect  to 
selection  of  TIS  and  LRF 

20.  Terrain 
grade 

a.  Level 

b.  Up  slope 

c.  Down  slope 

d.  Hilly 

Secondary  conditions 

21.  Terrain 
vegetation 

a.  None 

b.  Brush 

c.  Trees  -  scattered 

d.  Trees  -  dense  woods 

Secondary  conditions 

22.  Cultural 
features 

a.  Rural 

b.  Villages/towns 

c.  Suburban 

d.  Urban 

Secondary  conditions 

moving/stationary  distinction  fits  well  with  practical  considerations  for 
training  (e.g.  exercise  design  and  selection  of  tank  ranges).  However  one 
does  not  enter  a  battle  with  the  decision  to  fire  from  a  stationary  versus 
moving  tank,  but  rather  with  a  mission.  The  mission  is  the  primary 
determinant  of  own  vehicle  motion:  moving  primarily  for  offensive  missions 
and  stationary  primarily  for  defensive  mission.  Moving  to  a  short  halt,  a 
variation  of  moving,  may  be  used  depending  on  equipment  status  or  other 
conditions.  Similarly,  sight  used  and  engagement  technique  are  not 
conditions  themselves,  but  are  reactions  to  conditions.  That  is,  sight 
used  is  primarily  a  function  of  equipment  operational  status  and 
visibility.  These  appear  as  parameters  11  and  19.  Likewise,  engagement 
technique  is  a  function  of  other  parameters  including  target  range,  type, 
ammunition  loaded  at  the  time  of  contact,  and  equipment  status.  There¬ 
fore,  sight  and  engagement  technique  were  not  directly  adopted  as  gunnery 
parameters.  Rather,  the  conditions  that  determine  sight  and  engagement 
technique  were  included.  Our  operating  principle  was  to  identify  as 
parameters  only  those  factors  that  affect  gunnery  and  are  not  under  the 
immediate  control  of  the  Ml  crew.  Finally,  target  motion  and  number  of 
targets  are  presented  separately  as  parameter  2  (target  movement)  and 
parameter  3  (target  array).  A  time  factor  is  added  to  target  array 
because  number  of  targets  also  implies  some  time  interval  associated  with 
their  appearance. 

Of  the  eleven  conditions  identified  by  Kraemer  et  al.  (1975),  five 
were  not  used  in  our  analysis.  These  include  (1)  crewman  firing, 

(2)  weapon  used,  (3)  firing  mode,  (4)  fire  control  instrument,  and 
(5)  ammunition.  Given  that  tactical  context  perspective,  these  factors  do 
not  represent  environmental  conditions,  but  are  the  products  of  crew 
decisions  in  relation  to  the  environment.  For  example,  use  of  SABOT 
versus  HEAT  ammunition  depends  largely  on  the  type  of  target  to  be 
engaged,  and  secondarily  on  ammunition  supplies.  Target  type  and  supply 
levels,  but  not  ammunition  being  fired,  are  represented  in  our  conditions. 


Tactical  Gunnery  Behaviors 


Analysis  of  gunnery  behaviors  occurred  at  two  levels.  The  higher 
level  units  of  behaviors  are  referred  to  as  "activities"  and  are  roughly 
equivalent  to  the  term  "task."  The  former  term  was  adopted  to  distinguish 
between  the  outcome  of  the  present  analysis  and  the  official  list  of  armor 
tasks  that  has  been  formally  approved  by  the  Army.  The  subordinate  units 
of  analysis  are  called  "performance  elements"  or  simply  "elements"  which 
may  be  equated  to  steps  within  a  task.  These  two  levels  of  analysis  are 
detailed  in  the  following  sections. 


Activities.  Table  2-4  presents  the  activities  included  in  our 
definition  of  the  Ml  tank  gunnery  domain.  For  the  most  part,  activities 
were  derived  from  accepted  armor  doctrine.  For  instance,  Activities  3 
through  10  correspond  to  chapters  and  sections  within  chapters  of 
FM  17-12-1.  Activities  1  and  2  are  TM  activities  in  preparation  for 
firing.  Activity  11  (Assess  Results  of  Engagement)  relates  tactical 


Table  2-4 


Ml  Tactical  Gunnery  Activities 


Activities.  Parts,  and  Options  to  Gunnery  Behaviors 

ACTIVITY  1.  PREPARE  STATIONS  FOR  OPERATION  (PREOPS) 

ACTIVITY  2.  PERFORM  PREPARE-TO-FIRE  (PRE-FIRE)  CHECKS 
Option  2.1.  Prepare  for  Offense 
Option  2.2.  Prepare  for  Defense 

ACTIVITY  3.  ACQUIRE  TARGET(S) 

Part  3.1.  Search  for  Target(s) 

Option  3.1.1.  Search  Open  Hatch--Day 
Option  3.1.2.  Search  Closed  Hatch--Day 
Option  3.1.2.  Search  at  Night 
Part  3.2.  Detect/Locate/Identify  Target(s) 

Part  3.3.  Evaluate  Situation 


ACTIVITY  4.  ENGAGE  SINGLE  TARGET  WITH  THE  MAIN  GUN 

Option  4.1.  Engage  single  target  from  the  offense  using  precision  gunnery 
Option  4.2.  Engage  single  target  from  the  defense  using  precision  gunnery 
Option  4.3.  Gunner  cannot  identify  announced  target 
Option  4.4.  Engage  targets  using  TIS 

ACTIVITY  5.  ADJUST  FIRE 

Option  5.1.  Use  reengage  technique 
Option  5.2.  Use  standard  adjustment 
Option  5.3.  Use  TC  adjustment 

ACTIVITY  6.  ENGAGE  A  SINGLE  TARGET  WITH  THE  COAX 

ACTIVITY  7.  ENGAGE  MULTIPLE  TARGETS  WITH  THE  MAIN  GUN 

ACTIVITY  8.  ENGAGE  TARGETS  WITH  THE  CAL  .50  (INCLUDING  SIMULTANEOUS 
ENGAGEMENTS) 

Option  8.1.  Simultaneous  targets 
Option  8.2.  Cal  .50  targets 


ACTIVITY  9.  ENGAGE  TARGET  USING  DEGRADED  GUNNERY  TECHNIQUES 
Option  9.1.  Engage  target  using  battlesight  gunnery 

Option  9.2.  Engage  target  given  ineffective  LRF 

Option  9.3.  Engage  target  given  multiple  returns  from  LRF 

Option  9.4.  Engage  target  given  no  range  display  (loss  of  symbology) 

Option  9.5.  Engage  target  given  crosswind  sensor  failure 

Option  9.6.  Engage  target  given  cant  sensor  failure 

Option  9.7.  Engage  target  given  lead  angle  sensor  failure 

Option  9.8.  Engage  target  given  GPS  failure  (day  channel) 


(table  continues) 
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Option  9.9.  Engage  target  given  GPS/TIS  failure 
Option  9.10.  Engage  target  using  GAS 

Option  9.11.  Engage  target  given  stabilization  system  failure  (in 
emergency  mode) 

Option  9.12.  Engage  target  given  turret  power  failure  (in  manual  mode) 
ACTIVITY  10.  ENGAGE  TARGET(S)  FROM  THE  TC  POSITION 
ACTIVITY  11.  ASSESS  RESULTS  OF  ENGAGEMENT 
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considerations  discussed  in  FC  17-15.  Many  of  the  activities  are  further 
subdivided  into  "parts"  and  "options."  Parts  are  used  in  the  Target 
Acquisition  activity  to  divide  otherwise  sequential  elements.  Options  are 
used  in  several  activities  to  indicate  alternative  courses  of  actions  that 
are  dependent  on  various  equipment  conditions. 


r.v: 


Ml  gunnery  may  be  characterized  as  procedural  to  the  extent  that,  for 
any  given  set  of  conditions,  fairly  precise  steps  can  be  laid  out  for 
executing  an  engagement.  However,  there  are  an  enormous  number  of 
combinations  of  conditions  possible.  All  of  the  combinations  of  condi¬ 
tions  on  the  above  list  may  not  be  realistic,  so  it  would  be  very 
difficult  to  get  an  exact  count  on  the  number  of  possible  combinations. 
However,  a  very  rough  estimate  of  the  possible  combinations  of  conditions 
is  nearly  10  billion.-*-  Obviously,  the  differences  among  these 
combinations  will  vary  from  trivial  to  significant.  The  divisions  of 
activities  in  the  gunnery  domain  reflects  differences  in  conditions  which 
are  sufficient  to  alter  the  basic  procedure  by  more  than  just  a  few  steps. 
The  parameters  that  determine  which  activities  and  which  options  within 
activities  are  followed  are  the  primary  conditions  as  labeled  in 
Table  2-3.  For  example,  target  array  (parameter  4)  is  a  primary  condi¬ 
tions  because  it  determines  whether  an  engagement  is  for  a  single  target, 
multiple  targets,  or  simultaneous  targets  (activities  4,  7  and  8, 
respectively).  The  effects  of  the  remaining,  secondary  conditions  are 
more  subtle  and  are  not  apparent  in  the  divisions  of  the  activities.  For 
example,  parameter  5  is  target  cover  and  concealment.  Although  these  may 
affect  the  difficulty  of  detecting,  identifying,  and  hitting  targets,  the 
behaviors  executed  are  not  altered. 

The  activities  are  not  purely  sequential,  but  branch  and  loop  as 
indicated  in  Figure  2-1.  The  branching  connotes  decisions  made  by  the  TC 
based  on  what  kind  and  how  many  targets  are  detected  and  other  mission 
conditions.  Pre-operations  checks  and  prefire  checks  should  precede  all 
tactical  missions.  These  checks  include  evaluations  of  equipment, 
mission,  and  environment.  During  these  checks,  equipment  malfunctions  may 
be  discovered.  If  malfunctions  are  not  correctable,  degraded  mode 
techniques  may  be  called  for  in  continued  execution  of  the  mission. 

Figure  2-1  depicts  degraded  mode  techniques  as  a  box  between  pre-fire 
checks  and  target  acquisition.  These  degraded  mode  techniques  are  most 
easily  described  as  alterations  in  procedures  rather  than  as  different 
procedures.  Thus,  the  question  mark  in  the  box  and  the  placement  of  the 
box  in  Figure  2-1  should  be  interpreted  to  mean  that  various  alterations 
in  task  procedures  throughout  the  remainder  of  the  mission  should  be 


it 


1-This  was  calculated  as  the  product  of  the  following  conditions  per 
parameter:  Target  type  -  8;  Target  movement  -  5;  Target  cover  -  4;  Target 
array  -  3;  target  orientation  -  2;  Target  range  -  3;  Target  sector  -  3; 
IFFN  -  3;  Enemy  activity  -  6;  NBC  -  2;  Equipment  status  -  11;  Number  of 
crewman  -  2;  Supply  shortages  -  5;  Mission  and  formation  -  8;  Special 
engagement  techniques  -  3;  Visibility  -  4;  Terrain  grade  -  4;  Terrain 
vegetation  -  3.  Some  parameters  were  omitted  and  some  combined  to  avoid 
redundancy. 
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Figure  2-1.  Optional  sequences  for  performing  tactical  gunnery 


employed  if  equipment  malfunctions  occur.  Activity  9  in  Table  2-4  lists 
12  possible  degraded  mode  options. 

Target  acquisition  (Activity  3)  precedes  all  engagements.  Subsequent 
to  acquisition  are  five  types  of  engagements  that  are  sufficiently 
different  in  their  procedural  requirements  to  be  designated  as  separate 
activities.  These  include  single  target  main  gun  engagements 
(Activity  4),  coax  engagements  (Activity  6),  multiple  target  engagements 
(Activity  7),  caliber  .50  target  engagements  (Activity  8),  and  engagements 
from  the  TC  position  (Activity  10).  With  the  fire  control  system  of  the 
Ml,  there  is  not  sufficient  difference  in  behavior  requirements  to 
distinguish  between  moving  and  stationary  targets.  In  contrast,  movement 
of  the  firing  tank  is  noted  within  Activity  4  (Engage  Single  Target  with 
the  Main  Gun)  as  two  separate  activities  (Options  4.1  and  4.2).  To  avoid 
redundancy,  this  distinction  is  not  repeated  within  the  other  firing 
activities,  although  it  should  be  understood  that  these  options  are 
available  and  should  be  practiced. 

Activity  5  (Adjust  Fire)  is  sequenced  after  the  four  main  gun 
engagement  activities.  We  have  combined  target  evaluation  and  fire 
adjustment  because  they  are  closely  associated  elements  and  to  avoid 
redundancy  across  the  main  gun  engagement  techniques,  each  of  which  leads 
to  evaluation  of  target  hit  and  possible  fire  adjustment.  The  final 
tactical  gunnery  activity  is  Assess  Results,  which  is  viewed  as  a  transi¬ 
tion  activity.  In  Figure  2-1  it  is  assumed  the  mission  continues  and  the 
tank  crews  pick  back  up  with  target  acquisition.  Alternatively  the 
mission  may  end  with  this  activity. 

Performance  Elements.  The  detailing  of  performance  elements  within 
the  activities  was  conducted  concomitantly  with  the  specification  of 
activities.  Recall  we  argued  that,  given  a  set  of  conditions,  tactical 
gunnery  could  be  reduced  to  a  sequential,  albeit  lengthy,  procedure. 
However,  there  are  numerous  conditions  that  create  variations  in  the 
procedures.  One  of  the  reasons  for  creating  separate  activities  was  to 
accommodate  the  significant  differences  in  procedural  elements  neces¬ 
sitated  by  these  varying  conditional  parameters.  That  necessitated  the 
examination  of  elements  prior  to  settling  on  the  specification  of 
activities.  For  example,  some  activity  divisions  were  created  (e.g., 
firing  from  a  moving  versus  stationary  tank)  while  other  were  not  (e.g., 
firing  at  stationary  versus  moving  targets)  based  on  the  similarity  of 
procedures  necessary  under  these  different  conditions.  Similarly,  the 
exact  placement  of  boundaries  between  activities  resulted  from  joint 
considerations  of  the  major  goal  of  each  activity  and  the  commonalities 
between  elements  near  potential  boundaries  between  activities.  For 
example,  the  TC's  decision  to  engage  a  target,  and  his  selection  of  weapon 
and  crewman  were  placed  in  Activity  3  (Acquire  Targets)  to  allow  for  the 
branching  that  must  occur  as  a  result  of  these  decisions.  Similarly, 
while  "recovering  sight  picture"  is  the  behavior  that  immediately  follows 
firing,  it  is  most  logically  associated  with  the  goal  of  evaluating  target 
hit  and  was  placed  in  Activity  5  (Adjust  Fire).  Thus,  in  the  initial 
stages  of  analysis,  partitioning  the  gunnery  domain  into  activities, 
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parts,  and  options  and  detailing  the  elements  within  these  divisions  were 
iterative,  interacting  processes. 

Defining  the  tactical  gunnery  domain  in  broad  terms  necessitated  the 
use  of  a  variety  of  sources  of  information.  The  two  most  important 
sources  were  FC  17-12-1  and  TM  9-2350-255-10.  Kraemer's  (1983)  Ml  Crew 
Drills  provided  another  source  of  information.  Two  additional  sources 
provided  somewhat  non-traditional  elements  for  gunnery:  (a)  A  research 
report  by  Drucker,  Hannaman,  Melching,  and  O'Brien  (1983)  describes  non¬ 
procedural,  decision-making  elements  in  gunnery;  and  (b)  FC  17-15 
(Division  86  Tank  Platoon)  was  consulted  to  identify  behaviors  that  may 
not  be  strictly  related  to  "putting  steel  on  target,"  but  are  important 
for  platoon  coordination  and  occur  simultaneously  with  behaviors  more 
directly  related  to  marksmanship.  These  coordination  behaviors  can  create 
disruptive  time-sharing  problems  unless  they  are  incorporated  into  gunnery 
practice.  While  neither  type  of  behavior  may  be  difficult  by  itself, 
smooth  integration  occurs  only  with  practice.  In  order  to  keep  track  of 
these  integration  requirements,  they  were  explicitly  included  among  the 
behaviors  of  the  tactical  gunnery  domain. 

To  avoid  excess  redundancy,  two  rules  were  adopted.  First, 
procedures  that  are  normally  conducted  using  the  TM  (e.g.  boresight)  were 
not  laid  out  in  detail;  rather,  the  TM  was  referenced.  Second,  cross- 
referencing  between  activities  was  used.  The  first  rule  applied  to 
Activities  1  and  2;  the  second  rule  affected  activities  7  through  10. 
Regardless  of  such  formatting  short-cuts,  each  activity  still  represents 
an  internally  consistent  objective. 

The  final  tactical  gunnery  task  analysis  is  presented  in  Volume  II, 
Appendix  A.  The  basic  format  of  presenting  the  simultaneous  behaviors  for 
each  of  the  tank  crewman  in  four  columns  follows  the  convention  of  Kraemer 
(1983)  and  FM  17-12-1.  However,  the  analysis  is  more  inclusive  than 
either  source  in  both  depth  and  breadth:  (a)  in  depth  because  greater 
detail  is  provided  in  describing  behavior,  and  (b)  in  breadth  because 
elements  associated  with  activities  beyond  pure  gunnery  are  included. 


CHAPTER  3 


ARMOR  PERFORMANCE  DEFICIENCIES 


The  purpose  of  the  research  described  in  the  present  chapter  was  to 
identify  known  armor  performance  deficiencies  upon  which  we  would  focus 
the  device-based  training  program.  In  training  development,  performance 
deficiencies  are  commonly  identified  through  structured  interviews  of  and 
ratings  by  subject  matter  experts  (SMEs).  Having  conducted  many  similar 
analyses,  project  staff  members  were  wary  of  this  source  of  information 
because  of  its  inherent  subjectivity  and  unreliability.  Indeed,  we 
initially  compiled  a  rating  form  of  gunnery  skills  and  knowledges  for 
experts  to  estimate  (a)  the  percentage  of  crews  deficient  in  gunnery  skill 
and/or  knowledge  component,  and  (b)  the  importance  of  the  components  to 
overall  gunnery  performance.  Results  from  a  pilot  study  using  two  experts 
indicated  that  they  produced  highly  variable  estimates  of  percent  crews 
deficient  and  tended  to  rate  every  component  as  equally  important.  The 
other  problem  with  generating  our  own  survey  data  is  that  there  are  no  Ml 
TO&E  units  at  Fort  Knox.  Therefore,  there  is  a  local  shortage  of  experts 
who  are  familiar  with  performance  deficiencies  of  experienced  armor 
crewmen . 

We  subsequently  pursued  the  issue  of  task  performance  deficiencies 
using  three  other  information  sources  that  were  thought  to  yield  more 
objective  data.  First,  we  examined  survey  data  collected  by  the  Direc¬ 
torate  of  Evaluation  and  Standardization  (DOES).  Second,  we  conducted  a 
review  of  the  tank  gunnery  literature.  Third,  we  analyzed  Table  VIII 
performance  data  from  Grafenwohr.  The  findings  from  these  data  sources 
are  summarized  below. 


DOES  Survey  Data 

One  additional  reason  for  not  generating  our  own  survey  data  is  that 
the  DOES  maintains  related  data  for  their  own  purposes.  They  periodically 
survey  field  personnel  to  ascertain  the  quality  of  training  performed  by 
the  Armor  School.  Although  the  focus  of  these  surveys  is  on  recent  Armor 
School  graduates  rather  than  experienced  crewmen,  the  results  should 
nevertheless  provide  some  indication  of  the  relative  difficulty  of  gunnery 
tasks . 

No  one  course  is  directly  responsible  for  training  tank  gunnery. 

Tank  commander  skills  are  taught  in  the  Basic  Noncommissioned  Officer 
Course  (BNCOC);  however,  no  BNCOC  data  were  available.  Driver  and  loader 
skills  are  emphasized  in  entry-level  or  One  Station  Unit  Training  (OSUT) 
with  gunnery  skills  classified  as  "familiarization"  training.  That  means 
that  OSUT  graduates  are  not  expected  to  be  proficient  gunners.  Gunners' 
skills  are  learned  on-the-job  through  assignment  rotation  and  unit 
training.  Therefore,  no  one  course  represents  the  population  we  are  most 
interested  in  regarding  the  difficulty  of  gunnery  tasks.  OSUT  does, 
however,  offer  a  view  of  the  difficulty  of  driver  and  loader  tasks,  and 
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Perf  ormonce 


DOES  survey  data  were  available  for  OSUT.  Performance  of  gunner  tasks  is 
estimated  by  the  survey,  but  because  recent  OSUT  graduates  are  not  usually 
assigned  to  gunner  positions,  the  results  must  be  interpreted  cautiously. 

Supervisors  (n  =  23)  rated  recent  OSUT  graduates'  performance  level 
(cannot  perform,  less  than  adequate,  adequate,  more  than  adequate,  and 
exception)  and  performance  frequency  (never  performed,  seldom  performed, 
performed  monthly,  performed  weekly,  performed  daily)  for  Skill  Level  One 
Soldiers'  Manual  Tasks.  Task  means  for  these  two  ratings  were  highly 
correlated  (r  =  .69,  £<.01)  suggesting  that,  at  least  from  the  observa¬ 
tions  of  the  raters,  task  proficiency  is  dependent  on  frequency. 
Examination  of  the  frequency-by-performance  plot  suggested  a  curvilinear 
trend.  That  is,  the  incremental  effects  of  task  frequency  on  performance 
are  greatest  at  the  low  end  of  frequency  and  plateau  as  frequency 
increases.  This  was  tested  by  adding  a  curvilinear  component  (frequency 
squared)  to  a  multiple  regression  model  used  to  predict  performance 
ratings  from  frequency  ratings.  The  curvilinear  component  was  significant 
(£<.05),  increasing  the  prediction  of  performance  to  R=.72,  £<.01). 

Figure  3-1  depicted  the  relationship  between  frequency  and  performance. 
Each  cross  represents  a  task  and  the  curved  line  is  the  regression 
function. 


Figure  3-1.  Performance  level  as  a  function  of  performance  frequency 
for  68  armor  tasks. 
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Because  of  their  strong  relationship  with  task  frequency  ratings, 
performance  ratings  alone  are  not  a  good  indicator  of  task  difficulty. 

That  is,  tasks  may  have  low  performance  ratings  because  they  are  infre¬ 
quently  practiced.  Tasks  that  are  frequently  performed  and  also  have  low 
performance  ratings  would  be  of  more  interest.  In  order  to  obtain  a  task 
difficulty  index  independent  of  frequency,  residual  performance  was 
calculated  as  the  difference  between  performance  level  that  would  be 
predicted  by  the  regression  function  and  actual  performance  level. 

Residual  scores  near  zero  indicate  tasks  with  performance  levels  very 
close  to  what  is  expected  based  on  frequency  of  performance.  Residual 
scores  greater  than  zero  indicate  tasks  with  performance  levels  higher 
than  expected  ("easy"  tasks).  Residual  scores  less  than  zero  indicate 
tasks  with  performance  lower  than  expected  or  "hard"  tasks.  Task  means 
for  frequency,  performance  and  residual  performance  are  presented  in 
Appendix  B. 

Of  the  68  tasks  addressed  in  the  surveys,  18  tasks  fell  into  our 
domain  of  gunnery.  Only  three  of  these  tasks  were  in  the  lower  third  of 
the  distribution  of  residual  performance. ^  That  is,  three  gunnery  related 
tasks  were  performed  less  well  than  would  be  predicted  by  the  amount  of 
practice  they  received.  These  tasks  included,  beginning  with  the  most 
difficult:  (a)  identify  threat  aircraft,  (b)  identify  friendly  and  threat 
armored  vehicles,  and  (c)  drive  an  Ml  tank.  The  two  identification  tasks 
are  both  below  the  median  in  task  frequency  and  also  ir  raw  performance. 

On  the  other  hand,  driving  is  one  of  the  highest  frequency  tasks  (7th  out 
of  68).  However,  in  terms  of  performance,  it  ranks  23rd. 

On  the  other  end  of  the  spectrum,  12  of  the  18  tasks  fell  in  the 
upper  third  (easy)  portion  of  the  residual  performance  distribution,  with 
performance  better  than  expected.  Among  the  9  "least  difficult"  of  the  68 
tasks  were  the  following  gunnery  related  tasks:  (a)  engage  with  the  COAX, 
(b)  engage  with  the  main  gun,  (c)  zero  the  COAX,  and  (d)  engage  with 
loader's  M240.  While  each  of  these  tasks  fell  at  or  below  the  median  on 
performance,  the  level  is  commensurate  with  frequency  of  performance. 

^It  is,  of  course,  possible  that  the  residual  scores  represent  only 
random  variance,  and  that  their  rank  ordering  is  meaningless.  Because  the 
residual  scores  are  mean  scores,  it  is  possible  to  estimate  what  their 
variance  should  be  (by  chance)  from  the  variance  of  the  individual  level 
scores  from  which  they  were  derived.  To  do  this,  an  individual  level 
estimate  of  residual  variance  was  obtained  by  a  general  linear  regression 
model  predicting  performance  ratings  from  task  designations  (67  coded 
vectors)  and  individual  frequency  rating  data  (frequency  and  frequency 
squared)  across  the  1564  (63  tasks  x  23  persons)  performance  ratings. 

Mean  square  error  was  .824,  which  divided  by  67  degrees  of  freedom  for 
tasks,  gives  an  estimated  standard  error  of  the  mean  of  .012.  Observed 
variance  of  the  mean  residual  was  .055.  The  ratio  of  these  variances 
(4.58)  with  67  and  1358  degrees  of  freedom  is  statistically  significant 
p< .01,  indicating  real  differences  among  the  residual  means.  Interpreting 
the  extreme  thirds  of  the  distribution  is  therefore  reasonable. 


This  finding  suggests  that  there  are  not  likely  to  be  an  unusual  training 
problems  associated  with  these  tasks. 

These  data  are  interesting  both  for  what  they  suggest  are  difficult 
tasks  and  what  they  suggest  are  not  difficult  tasks.  Aircraft  and  vehicle 
identification  are  well  known  as  difficult  tasks  (e.g.,  Warnick  &  Kubala, 
1979;  Knerr  et  a  1 . ,  1986).  In  addition  to  their  difficultly,  they  are 
relatively  infrequently  practiced  tasks  (less  than  monthly).  Certainly 
there  would  be  some  advantage  to  integrating  target  identification  into 
other  type  of  practice,  most  obviously  gunnery  exercises.  For  driving, 
there  may  be  a  difference  in  the  kind  of  driving  most  frequently  done 
(e.g.  to  and  from  the  motor  pool)  and  the  tactical  driving  needed  for 
quality  performance. 

The  gunnery  engagement  skills  were  not  singled  out  as  deficient  by 
our  residual  performance  method.  Rather  they  were  described  as  on  par 
with  frequency.  In  other  words,  there  were  no  deficiencies  in  gunnery 
performance  that  were  not  also  associated  with  infrequent  in  practice. 
Thus,  the  issue  for  training  is  not  the  difficulty  of  the  tasks  but  the 
opportunity  to  practice.  The  Skill  Level  One  Soldiers,  on  whom  the 
ratings  were  focused,  may  be  expected  to  improve  their  gunnery  skills  with 
additional  practice. 


Empirical  Research  Literature 


Unlike  most  other  job  domains,  there  is  considerable  empirical 
research  on  armor  gunnery  performance.  To  review  this  extensive  litera¬ 
ture,  the  domain  of  gunnery  was  broken  down  into  a  number  of  broad, 
generally  recognized  categories  of  armor  skills  and  knowledges.  In  that 
regard,  the  literature  on  armor  job  samples^  was  examined  particularly 
closely  since  many  armor  job  samples  are  addressed  specifically  to  these 
gunnery  skills  and  knowledges  components.  Unfortunately,  the  literature 
has  not  directly  addressed  the  issue  of  identifying  performance  deficien¬ 
cies.  Nevertheless,  some  findings  have  addressed  issues  that  are  related 
to  task  criticality.  The  following  questions  are  basic  to  the  issues  at 
hand: 


1.  Is  there  evidence  for  any  relationship  between  the  skill  or 
knowledge  component  and  gunnery  performance?  Positive  correlations  may  be 
interpreted  as  evidence  that  a  particular  component  is  critical.  The 
major  problem  in  answering  this  question  is  the  criterion  of  gunnery 
performance.  In  most  of  the  literature  surveyed,  the  criterion  was 
performance  <•  i  a  gunnery  range;  in  fewer  cases,  other  criteria  (e.g., 
performance  ,.1  training)  was  used.  Gunnery  performance  was  chosen  most 


^Job  sample  tests  are  abstracted  portions  of  a  particular  job  on 
which  performance  may  be  scored.  Job  sample  test  scores  are  usually  used 
to  predict  performance  on  the  job  for  purposes  of  personnel  selection  and 
classification,  although  Kress  (1981)  used  the  principle  of  job  sampling 
to  design  a  training  program  for  armor  crewmen. 


often,  because  it  is  the  most  obvious  and  face  valid  criterion  for  gunnery 
proficiency.  Unfortunately,  gunnery  performance  is  notoriously  unreliable 
(e.g.,  Powers,  McCluskey,  Haggard,  Boycan,  &  Steinheiser,  1975).  Gunnery 
may  be  more  reliably  measured  by  high  fidelity  simulators  (e.g.,  Graham 
1986),  however  research  in  this  area  has  only  recently  begun  to  examine 
these  and  related  issues. 

2.  Is  the  skill  or  knowledge  component  trainable?  In  most  cases, 
the  available  research  findings  were  correlational  in  nature. 

Consequently,  we  cannot  determine  whether  the  relationships  between 
sk i 1 1 /knowledge  and  gunnery  performance  are  due  to  differences  in  achieve¬ 
ment  or  aptitude.  Because  our  interest  is  in  training  gunnery  skills,  the 
focus  of  the  present  review  is  on  those  skills  that  are  accumulated  as  a 
result  of  experience  rather  than  those  that  distinguish  innately  good  and 
bad  performers.  Skill  and  knowledge  are  generally  regarded  as  trainable 
if  performance  on  the  components  can  be  shown  to  increase  with  practice  or 
with  relevant  experience.  We  may  also  ask  whether  or  not  the  acquired 
skills  transfer  from  the  training  situation  to  the  operational  context. 
Unfortunately,  the  literature  has  not  addressed  this  issue  well.  (For  a 
methodo’ 'gical  critique,  see  Boldovici,  in  press.) 

The  details  of  the  literature  review  are  provided  in  Appendix  C,  and 
the  findings  are  summarized  in  Table  3-1.  A  table  entry  of  "yes" 
indicates  that  there  is  good  evidence  for  an  affirmative  answer  to  one  or 
the  other  research  question.  "Maybe"  denotes  that  the  findings  from  the 
research  literature  are  mixed,  but  at  least  one  study  indicates  that  the 
component  is  correlated  with  some  aspect  of  gunnery  proficiency.  "No 
research"  simply  indicates  that  no  empirical  research  has  not  been 
addressed  to  the  research  question.  With  regard  to  the  final  entry, "no 
evidence,"  it  is  important  to  note  that  we  cannot  conclude  from  a  lack  of 
positive  findings  that  an  effect  does  not  exist. ^  For  instance,  it  would 
be  wrong  to  conclude  that  the  knowledge  and  skill  components  related  to 
adjustment  of  fire  are  not  significant.  Two  objections  may  be  raised 
against  such  a  conclusion.  First  of  all,  as  stated  earlier,  the  criterion 
for  gunnery  proficiency  (performance  on  Table  VIII)  may  have  been 
unreliably  measured,  effectively  preventing  any  score  to  correlate  with 
gunnery.  Second,  performance  on  the  skill  and  knowledge  component  may 
have  been  inval idly  or  unreliably  measured. 

We  could  confirm  only  one  component  of  gunnery  (observational  skill) 
that  correlated  with  gunnery  performance.  In  another  context,  a  similar 
lack  of  findings  was  reported  by  Means  and  Gott  (1986)  who  found  no 
differences  between  more-  and  less-expert  Air  Force  troubleshooters  in 
basic  electronics  knowledge  when  tested  in  isolation.  However,  they  found 
important  deficiencies  in  basic  knowledges  when  the  technicians  performed 
actual  complex  troubleshooting  tasks.  Apparently,  the  isolated  knowledges 
were  somehow  different  from  the  knowledges  in  the  job  context.  The 


4In  the  practice  of  statistical  analysis,  this  error  in  reasoning  is 
referred  to  as  "accepting  the  null  hypothesis." 


_ Research  Questions _ 

Related  to  Gunnery 

Hypothesized  Component  Proficiency?  Trainable? 


Target  Acquisition 


Crew  Search 

No  research 

No  research 

Detection 

Maybe 

Yes 

Location 

No  research 

No  research 

Identification 

Maybe 

Yes 

Classification 

No  evidence 

No  evidence 

Range  Estimation 

No  evidence 

No  evidence 

Knowledge  of  Fire  Control  System 

No  research 

Maybe 

Firing  the  First  Round 

Sight  Picture/ 

No  evidence 

No  research 

Stationary  Engagements 

Maybe 

Yes 

Track  ing/ 

Maybe 

Yes 

Moving  Engagements 

Maybe 

Maybe 

Firing  Subsequent  Rounds 

Observations 

Yes 

No  research 

Fire  Adjustments 

No  evidence 

No  research 

Procedural  Skills  and  Knowledges 

Technical  Manual 

No  evidence 

No  research 

Fire  Control  Computer 

Maybe 

No  research 

authors  concluded  that  if  troubleshooting  is  the  targeted  skill,  it  should 
be  studied  directly,  rather  than  by  examining  tests  that  correlate  with 
troubleshooting  performance.  Their  conclusions  suggested  that  we  need  to 
examine  other  data  sources  that  measure  gunnery  performance  per  se.  One 
such  source  is  the  Grafenwohr  gunnery  data  base  that  is  maintained  by  the 
Office,  Chief  of  Armor  (OCOA).  This  data  base  provides  much  information 
on  the  performance  of  armor  crews  on  Table  VIII. 


Grafenwohr  Gunnery  Data  Base 

Table  VIII  is  a  live-fire  gunnery  test  designed  to  determine  whether 
or  not  individual  crews  are  qualified.  The  Office,  Chief  of  Armor  (OCOA) 
maintains  a  detailed  data  base  on  Table  VIII  performance  at  Grafenwohr, 
one  of  several  sites  at  which  Table  VIII  is  administered.  The  data  base 
provides  information  for  the  Chief  of  the  Armor  Branch  and  serves  as  a 
source  of  research  data  for  other  Army  organizations.  The  data  base  is  a 
rich  and  complex  source  of  information  on  gunnery  performance,  including 
information  about  the  crewmembers,  the  targets,  and  the  test  conditions. 

In  many  ways,  this  data  base  provides  an  ideal  source  on  information  about 
gunnery  performance  deficiencies. 

Grafenwohr  Table  VIII  data  from  January  -  July  1986  were  manipulated 
to  answer  a  list  of  specific  questions  about  gunnery  performance.  Details 
of  the  analysis  and  the  findings  are  presented  in  Appendix  D.  The  results 
did  not  reveal  many  specific  performance  deficiencies  that  could  be 
addressed  by  the  proposed  device-based  training/testing  program.  Never¬ 
theless,  the  analysis  did  provide  some  findings  that  may  be  of  interest  to 
the  armor  community.  These  findings  may  be  summarized  as  follows: 

1.  The  data  failed  to  confirm  expected  differences  between  moving 
vs.  stationary  targets,  offensive  vs.  defensive  engagements,  and  precision 
vs.  battlesight  engagements. 

2.  Performance  was  slower  and  less  accurate  for  distant  and/or  small 
targets,  but  this  relationship  is  probably  due  to  both  human  and  machine 
error. 

3.  Gunnery  performance  was  dramatically  reduced  when  crews  engaged 
targets  in  an  NBC  environment. 

4.  Crew  cuts  associated  with  the  fire  commands  were  the  most  likely 
observed  procedural  errors.  Furthermore,  errors  in  fire  commands  were 
associated  with  poorer  gunnery  performance,  although  few  of  the 
comparisons  were  statistically  reliable. 

5.  Similar  differences  between  engagements  that  were  observed  in  the 
Grafenwohr  data  base  were  also  observed  in  findings  from  other  Table  VIII 
sites,  supporting  the  generality  of  the  findings. 


29 


Conclusions 


These  various  data  sources  did  not  allow  us  to  identify  a  definitive 
set  of  performance  deficiencies.  As  a  result  of  these  negative  results, 
we  are  left  with  two  plausible  interpretations  that  cannot  be  confirmed  or 
denied  by  the  data  alone.  One  interpretation  is  that  our  methods  were  not 
sensitive  enough  to  detect  performance  differences  between  components  of 
armor  gunnery.  The  subjective  weight  of  the  evidence  across  three 
different  sources  of  information  would  tend  to  deny  this  interpretation. 
Furthermore,  it  offers  no  implication  for  the  design  of  our  training 
program. 

The  second  interpretation  that  may  be  derived  from  these  results  is 
that  there  are  actually  no  outstanding  differences  between  components  in 
performance  difficulty  or  criticality.  Given  the  pilot  SMEs  subjective 
judgments  that  everything  was  important,  one  implication  from  this 
conclusion  is  that  every  component  of  gunnery  is  critical  for  training. 

In  a  sense,  this  must  be  true  given  the  highly  technical  nature  of  armor 
gunnery.  With  the  exception  of  very  few  generic  skills  such  as  ability  to 
read  technical  manuals,  an  armor  recruit  enters  training  possessing  none 
of  the  skills  and  knowledges  related  to  gunnery.  Furthermore,  in  order  to 
perform  gunnery,  particularly  tactical  gunnery  as  we  have  defined  it,  he 
must  acquire  the  entire  domain  to  perform  successfully.  Thus,  every 
component  is  critical  for  training.  The  latter  is  the  conclusion  we 
reached  for  gunnery.  Consequently,  the  proposed  device-based  training 
program  shall  consider  the  entire  domain  of  tactical  gunnery  rather  than 
any  subset  of  that  domain. 
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CHAPTER  4 


INSTRUCTIONAL  ANALYSIS  AND  PERFORMANCE  MEASUREMENT 
APPROACHES  TO  Ml  TACTICAL  GUNNERY 


In  Chapter  2,  the  performance  domain  of  Ml  tactical  gunnery  was 
presented  as  a  series  of  interrelated  activities.  In  the  training 
context,  these  activities  represent  the  initial  partitioning  of  our 
terminal  training  objective,  tactical  gunnery,  into  subordinate 
objectives.  The  purpose  of  this  chapter  is  to  elaborate  on  our  conceptual 
organization  of  these  behaviors.  The  chapter  is  divided  into  two 
sections:  The  first  describes  the  hierarchical  analytic  approach  used  to 
identify  training  and  testing  objectives;  the  second  outlines  our 
performance  measurement  approach  for  assessing  performance  within  the 
training  context.  The  hierarchies  presented  in  this  chapter  partition  all 
activities  in  the  performance  domain  into  training  objectives  and  indicate 
sequences  of  instruction.  They  also  identify  any  prerequisite  objectives. 
The  hierarchies  thus  represent  rudimentary  training  plans.  As  such,  they 
provide  the  starting  point  for  the  device  evaluations  leading  to  the 
recommendations  for  device  use  in  gunnery  training  and  testing. 


Hierarchical  Analysis  Approach 


A  learning  hierarchy  is  a  two-dimensional  representation  of  the 
super-  and  subordinate  relationships  between  training  objectives.  As 
originally  defined  by  Gagne  (1962),  a  subordinate  relationship  indicates 
that  a  training  objective  is  prerequisite  to  another  objective.  That  is, 
in  order  for  a  student  to  perform  at  a  particular  skill  level,  he  must 
possess  all  ubordinate  skills  specified  by  the  hierarchy.  White  (1973) 
reviewed  the  evidence  in  favor  of  hierarchical  ordering  and  found  that  a 
majority  of  students  demonstrate  the  skills  in  the  predicted  order. 
However,  there  were  also  significant  numbers  of  students  who  possessed  the 
higher  level  skills  without  possessing  all  subordinate  skills  as  specified 
by  the  hierarchy.  Cotton,  Gallagher,  and  Marshall  (1977)  noted  that  one 
possible  reason  for  failing  to  find  strict  hierarchical  ordering  is  that 
some  types  of  tasks  are  not  amenable  to  hierarchical  analysis.  In  that 
view,  Gagne  (1968)  distinguished  between  intellectual  skills  ( i . e . ,  tasks 
that  are  procedural  in  nature)  and  verbalized  knowledge  (i.e.,  tasks 
requiring  the  recall  of  information  or  facts)  with  hierarchical  analysis 
techniques  applying  to  the  former  and  not  the  latter.  Nevertheless, 

Cotton  et  al.  concluded  that  some  subordinate  skills  "...may  simply 
transfer  to  the  next  higher  item  rather  than  being  fully  prerequisite  to 
it"  (p.  189).  This  assumption  appears  to  be  tacitly  accepted  in  the  more 
recent  hierarchical  analysis  techniques  described  by  Resnick  (1976).  She 
distinguished  between  skills  "...that  are  thought  to  be  either  necessary 
to  performance  (i.e.,  prerequisite  to)  or  helpful  in  learning  (i.e., 
propadeutic  to)  the  main  task"  (p.  66).  Thus,  it  appears  reasonable  to 
redefine  hierarchically  subordinate  skills  as  either  prerequisite  or 
propadeutic  to  the  immediately  superior  skill  in  the  hierarchy. 


Research  on  learning  hierarchies  has  been  criticized  on  grounds  other 
than  the  prerequisite  assumption.  For  instance,  White  (1973)  found  the 
studies  of  learning  hierarchies  to  be  methodologically  inadequate.  Others 
have  noted  the  absence  of  an  appropriate  statistical  model  for  testing 
hierarchical  assumptions  (Bergan,  1980).  Possible  alternative  models  have 
been  suggested  such  as  Markov  models  (Cotton  et  a  1  . ,  1977),  path  analytic 
models  (Bergan,  1980),  and  even  latent  class  models  (Bergan  &  Stone, 

1985).  Despite  the  criticisms  of  research  and  theoretical  alternatives  to 
learning  hierarchies,  no  model  has  supplanted  hierarchical  analysis  as  a 
pragmatic  tool  for  the  design  of  training  and  testing.  To  that  point, 
Gagne  and  Dick  (1983)  reviewed  two  large-scale  projects--one  in  early 
mathematics  education  and  one  in  military  training--that  employed 
hierarchical  analysis.  These  authors  concluded  that  hierarchical  analytic 
techniques  significantly  contributed  to  the  success  of  these  projects. 


Example  Analysis:  Introductory  Mathematics 

An  oft-cited  and  highly  detailed  example  of  hierarchical  analysis  was 
provided  by  Resnick,  Wang,  and  Kaplan  (1973)  who  presented  learning 
hierarchies  for  a  preschool  mathematics  education  program.  An  example 
learning  hierarchy  for  the  skills  related  to  counting  is  presented  in 
Figure  4-1.  The  domains  of  arithmetic  and  gunnery  are  similar  in  several 
respects.  For  one,  both  are  governed  by  explicit  procedural  rules. 
Although  there  is  a  component  of  perceptual  motor  skill  to  gunnery 
performance,  procedural  task  skills  appear  to  predominate  over  fine  motor 
ski  1 ls--especially  with  the  advent  of  the  many  automated  components  of  the 
Ml.  Furthermore,  both  domains  consist  of  a  finite  number  of  practice 
elements,  i.e,  "exercises"  in  math  and  "engagements"  in  gunnery.  This 
characteristic  makes  task  sampling  relatively  easy  and  straightforward. 
Because  of  these  similarities  between  domains,  the  procedures  used  by 
Resnick  et  al.  were  examined  and  modified  for  application  to  the  present 
project.  The  modified  procedures  are  summarized  below: 

1.  Hierarchies  are  diagrammed  as  inverted  tree  structures.  Every 
learning  objective  is  identified  in  a  box  and  the  boxes  are  arranged  to 
indicate  their  subordinate  relationships.  Terminal  objectives  are  drawn 
at  the  top  of  the  diagram;  lowest  level  enabling  objectives  are  drawn  at 
the  bottom.  Enabling  objectives  are  connected  to  the  superordinate 
objectives  they  support. 

2.  The  analysis  generally  proceeds  from  the  top  down.  That  is,  each 
terminal  objective  is  analyzed  to  reveal  subordinate  objectives  until  a 
skill  level  is  reached  that  may  be  assumed  to  be  possessed  by  all  members 
of  the  to-be-trained  population.  However,  as  analyses  proceed  downward, 
unexpected  similarities  or  dissimilarities  among  elements  may  be 
uncovered,  necessitating  changes  in  the  partitioning  of  upper  level 
divisions.  Thus,  while  the  analysis  generally  proceeds  along  the  vertical 
dimension,  it  is  accompanied  by  elaboration  along  the  horizontal  dimension 
as  well. 
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Fixed  ordered  set 
of  objects 


Count  objects. 


Fixed  ordered  set 


Touch  first  object 
and  say  first 
numeral  ("one"). 


Fixed  ordered  set 


Touch  next  object 
and  say  next 
numeral. 


When  last  object 
has  been  touched 


State  last  numeral 
as  number  in  set. 


Set  of  objects 


Synchronize  touching 
object  and  saying 
a  word. 


Touch  an  object  or 
tap  each  time 
word  is  stated. 


Recite  numerals 
in  order. 


Repeated  tap  or  touch 
by  another  person 
Say  a  word  each 
time  there  is  a  tap. 


Fixed  set  of  objects 
Touch  each  object 
once  and  only  once 
(i.e.,  "remember” 
which  objects  have 
been  touched). 


Row  of  objects 

Touch  each  object 
in  order  beginning 
at  an  end  of 
the  row. 


Figure  4-1.  Example  learning  hierarchy  of  counting  task 

(from  Resnick,  Wang,  &  Kaplan,  1973.  Reprinted 
with  permission). 


3.  The  hierarchy  is  behaviorally  based.  Information  in  each  box 
includes  the  stimulus  situation  (above  the  vertical  line)  and  appropriate 
responses  (below  the  vertical  line)  associated  with  each  training  and 
testing  objective.  This  format  is  compatible  with  the  Army's  philosophy 
toward  training  and  testing  that  promotes  explicit  statements  of 
conditions  and  actions  for  each  training  and  testing  objective. 

4.  The  hierarchy  provides  a  formal  model  of  performance.  Resnick  et 
al.'s  method  is  consistent  with  production  systems  models  of  performance. 
Productions  are  statements  consisting  of  an  action  paired  with  conditions 
under  which  the  action  is  to  be  performed.  Production  systems  have  been 
used  to  model  overt  procedural  performance  (e.g.,  Sticha,  1987;  Knerr  et 

a  1 . ,  1986)  as  well  as  covert  mental  processes  (Newell  1973;  Anderson, 
1982).  The  implication  is  that  this  approach  can  model  behavioral  as  well 
as  mental  processes. 

5.  Arrows  are  used  to  indicate  sequential  dependencies  between  overt 
behaviors.  Loops  are  used  to  indicate  that  performers  must  recycle 
through  certain  series  of  steps.  Decision  points  are  represented  by 
splitting  the  box  vertically  to  indicate  different  stimulus-response 
contingencies. 

6.  Prerequisites  skills  are  indicated  below  the  level  of  overt 
behaviors.  These  prerequisite  skills  are  not  actually  performed  in  the 
course  of  the  terminal  behavior  but  are  assumed  to  facilitate  learning  of 
the  higher  level  skill.  Subordinate  skills  may  be  further  analyzed  into 
lower  order  components  until  the  lowest  level  of  skill  is  reached.  Note, 
however,  that  even  the  prerequisite  skills  are  behaviorally  defined  by 
providing  a  statement  of  specific  conditions  and  actions  that  apply. 


Example  Analysis:  Battlesiqht  Gunnery 

To  illustrate  how  the  analytic  procedures  described  by  Resnick  et  al. 
(1973)  were  incorporated  and  modified  in  the  current  project,  Figure  4-2 
presents  an  analysis  of  Option  9.1:  Engage  target  using  battlesight 
gunnery.  This  activity  was  chosen  as  an  example,  because  it  is  somewhat 
simpler  than  most  objectives  in  the  gunnery  domain  thereby  facilitating 
its  exposition.  Nevertheless,  the  example  illustrates  most  of  the 
features  of  the  analysis  method.  The  analyses  for  remainder  of  the 
gunnery  domain  are  contained  in  Appendix  E. 

One  of  the  most  salient  distinctions  between  mathematics  and  gunnery 
performance  is  that  gunnery  tasks  not  only  require  individual  perform¬ 
ances,  but  the  collective  interaction  of  the  individuals  acting  as  a  crew. 
As  a  result,  individual  behavioral  objectives  must  be  clarified  by  using 
the  crewman  position  (TC,  GNR,  LOR,  or  DVR)  as  the  subjects  of  the  action 
statements.  If  no  subject  is  provided,  it  is  understood  that  the  entire 
crew  performs  the  action.  At  the  prerequisite  level,  subjects  are  not 
provided  implying  that  all  crewmen  should  possess  these  basic  skills.  The 
crewman  position  name  does  not  imply  that  he  alone  should  be 
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trained  on  the  objective.  On  the  contrary,  we  assume  that  students  in  the 
course  will  be  cross-trained  on  all  skills. 


Level  I  of  Figure  4-2  corresponds  to  the  crew-level  objective  of 
engaging  a  target  using  battlesight  gunnery.  As  illustrated  in  the 
figure,  some  conditions  and  actions  were  best  described  as  combinations  of 
simpler  stimuli  or  responses.  The  logical  operators  and  "OR"  were 
used  to  connect  the  conditions.  The  operators  were  underlined  to  set  them 
off  from  the  rest  of  the  text  within  the  boxes.  In  the  present  case,  the 
Level  I  objective  indicates  that  battlesight  gunnery  is  appropriate  in 
either  a  surprise  situation  or  when  the  LRF  is  ineffective.  However,  the 
target  must  be  within  battlesight  range  in  either  case. 

Level  II  of  this  figure  corresponds  to  the  behavioral  component  of 
the  task.  The  behavioral  objectives  are  outlined  by  noticeably  thicker 
lines  to  distinguish  them  from  crew-level  and  prerequisite  objectives. 5 
In  the  present  analysis,  the  tank  commander's  fire  command  occurs  as  a 
result  of  the  preceding  acquisition  process.  Therefore  the  initiating 
condition  for  both  Objectives  1 1  a  and  1 1 b  is  the  battlesight  fire  command. 
Note  that  the  arrows  indicate  that  the  first  two  behavioral  objectives  are 
not  performed  sequentially,  but  rather  occur  simultaneously.  Objective 
1 1  a  calls  for  the  TC  to  use  the  manual  range  controls  to  index  battlesight 
range  into  the  ballistic  computer,  whereas  Objective  I I b  relates  to  the 
gunner  setting  the  gunner  panel  switches  for  a  battlesight  engagement. 
After  the  first  two  behavioral  objectives  are  accomplished,  the  analysis 
indicates  that  the  soldier  should  continue  to  behave  as  outlined  in 
Activity  4:  Engage  Target  Using  Precision  Gunnery.  The  dashed  lines  used 
to  enclose  Objective  lie  indicate  that  the  objective  has  been  previously 
analyzed  in  detail.  This  convention,  which  was  used  by  Resnick  et  a  1 . , 
reduced  unnecessary  redundancy  across  analyses. 

Note  that  Objectives  1 1  a  and  1 1 b  consist  of  multiple  performance 
elements.  For  example,  Objective  I I b  (the  act  of  setting  gunner  switch 
positions)  consists  of  the  following  behavioral  elements:  Ensure  that 
(a)  FIRE  CONTROL  switch  is  in  NORMAL  position,  (b)  LRF  is  in  SAFE,  (c)  GPS 
magnification  is  on  3X,  (d)  GUN  SELECT  is  in  MAIN  position,  and  (e)  AMMO 
SELECT  is  set  to  the  predetermined  battlecarry  ammo.  We  determined  during 
the  process  of  analysis  that  making  individual  training  objectives  for 
such  isolated  elements  is  absurd.  Rather,  the  rule  we  followed  was  that 
training  objectives  were  created  for  meaningful  chunks  or  units  of 
behaviors.  These  meaningful  "units"  are  similar  to  the  concept  of 
behavior  chunks  (Miller,  Galanter,  &  Pribram,  1960),  which  have  more 
recently  been  related  to  the  concept  of  production  systems  (Anderson, 

1982).  Anderson  argued  that  the  performance  of  novices  may  be 


^Resnick  et  al.  distinguished  the  behavioral  components  from  the 
other  levels  of  the  hierarchical  analysis  by  using  double  lines  to  outline 
boxes.  In  our  example  and  in  Appendix  E,  we  use  thicker  lines  for  the 
same  purpose.  This  variation  in  representation  was  caused  by  the  fact 
that  our  graphics  software  could  not  easily  produce  double  lines. 


36 


characterized  by  productions  consisting  of  single  sets  of  conditions  and 
actions.  In  contrast,  expert  performance  is  better  simulated  by  macro 
productions  consisting  of  multiple  sets  of  conditions  and  multiple 
responses.  Although  the  training  objectives  presented  in  this  report  are 
conceptually  similar  to  such  macro  productions,  the  training  objectives 
were  derived  from  a  rational  analysis  of  gunnery  rather  than  from  an 
empirical  analysis  of  expert  performance. 

Finally,  Level  III  of  Figure  4-2  comprises  the  prerequisites  to  task 
performance.  According  to  the  hierarchy,  before  learning  how  to  index 
battlesight  range  at  the  TC's  station  (Objective  1 1  a ) ,  the  student  must 
know  how  to  (a)  visually  estimate  the  range  to  the  target  (Objective  Ilia) 
and  (b)  operate  the  manual  range  controls  (Objective  1 1 1 b ) .  Similarly, 
before  learning  how  to  set  the  gunner  panel  switches  for  a  battlesight  * 
engagement  (Objective  1 1 b ) ,  the  student  must  know  the  correct  switch 
settings  for  a  variety  of  engagement  scenarios  (Objective  1 1 1 c ) .  Whereas 
the  prerequisite  skills  are  not  observable  in  performance  of  the  task 
itself,  the  prerequisites  are  written  in  behavioral  fashion  such  that 
performance-oriented  training  and  testing  is  applicable  for  prerequisites 
as  well  as  for  training  overt  components  of  the  task. 


Conclusions 

In  general,  the  gunnery  domain  may  be  characterized  as  wide  but  not 
very  deep,  especially  compared  to  the  mathematics  examples  presented 
earlier.  By  wide,  we  mean  that  the  student  must  know  how  to  perform  a 
wide  range  of  disparate  activities.  By  its  lack  of  depth,  we  mean  that 
performance  is  not  dependent  on  a  lot  of  prerequisite  skills  and 
knowledges.  These  structural  characteristics  of  the  gunnery  domain  have 
some  general  implications  for  the  acquisition  and  retention  of  gunnery 
skills.  For  one,  initial  train-up  of  gunnery  skills  should  be  relatively 
fast  since  training  is  not  dependent  on  complex  layers  of  prerequisite 
skills.  However,  skill  sustainment  may  be  a  problem  for  two  reasons: 

(a)  the  sheer  extent  and  diversity  of  the  domain,  and  (b)  the  predominance 
of  procedural  tasks  which  research  has  indicated  are  especially  suscep¬ 
tible  to  decay  (e.g.,  Schendel,  Shields,  &  Katz,  1978). 

The  hierarchical  analyses  of  gunnery  were  performed  for  some  specific 
purposes:  to  provide  a  sequence  for  training  gunnery  objectives,  to 
provide  a  testing  strategy,  and  to  integrate  device  training  and  testing. 
The  following  subsections  discuss  those  purposes  and  some  of  the  problems 
that  the  gunnery  domain  structure  poses  for  design  of  the  proposed 
training/testing  program. 

Training  Sequence.  The  concept  of  the  learning  hierarchy  was 
originally  developed  to  identify  and  sequence  intellectual  skills  for 
instruction.  The  sequencing  rule  is  simple:  insure  that  prerequisite 
skills  are  trained  prior  to  moving  to  a  superordinate  objective.  In  other 
words,  instructional  sequencing  may  be  characterized  as  building  skills 
from  the  "bottom-up".  For  instance,  it  does  not  matter  whether  the 
student  learns  first  to  estimate  target  range  or  to  operate  TC's  manual 


range  controls.  However,  both  prerequisite  skills  must  be  trained  before 
training  the  TC's  duties  in  battlesight  gunnery.  Thus,  the  hierarchy 
identifies  constraints  to  sequence,  but,  as  shown  in  the  example,  it  does 
not  prescribe  a  single  correct  order  of  instruction. 

Because  of  the  width  of  the  gunnery  domain,  hierarchical  analyses 
specifies  only  a  proportionally  small  number  of  required  sequences  in 
instruction.  Additional  rules  are  needed  to  sequence  instruction.  One 
source  of  additional  sequencing  rules  is  a  strategy  that  may  be  termed 
"progressive  elaboration"  (D.  W.  Bessemer,  personal  communication, 

December  1987).  According  to  this  strategy,  students  start  training  on 
some  basic  procedure  without  elaboration  such  as  the  condition/action 
options  described  in  the  gunnery  domain.  These  elaborations  are  added 
only  after  the  basic  procedure  is  well  learned.  Furthermore,  prere¬ 
quisites  for  the  more  elaboration  procedures  should  be  trained  just  prior 
to  the  procedure  that  they  apply  in  order  to  minimize  retention 
decrements.  In  the  present  domain,  precision  gunnery  provides  a  basic  and 
central  gunnery  procedure  around  which  a  progressive  elaboration  strategy 
can  be  based. 

Diagnostic  Testing  Strategy.  Learning  hierarchies  have  also  been 
used  to  develop  a  diagnostic  strategy  (Wang,  1973).  For  instance, 
proficiency  within  a  domain  may  be  measured  by  testing  appropriate  high 
level  skills.  Passing  a  particular  testing  objective  implies  that 
connected  subordinate  objectives  need  not  be  tested.  On  the  other  hand, 
failing  one  of  objectives  would  require  subsequent  testing  of  lower  order 
skills  to  identify  the  locus  of  the  performance  deficiency.  Thus,  a  "top- 
down"  approach  to  testing  can  be  used  to  efficiently  identify  the  locus  of 
any  performance  deficiency.  The  relative  flatness  of  the  gunnery  domain 
limits  the  applicability  of  this  approach;  nevertheless,  it  should  still 
be  considered.  Testing  strategy  is  discussed  more  fully  in  the  second 
half  of  the  chapter. 

Evaluation  and  Integration  of  Training  Devices.  The  hierarchical 
analysis  will  identify  discrete  units  of  learning  that  can  be  concept¬ 
ualized  as  building  blocks  for  acquiring  proficiency  for  the  whole  domain. 
These  blocks  also  represent  independent  units  for  device  evaluations.  That 
is,  instead  of  evaluating  devices  relative  to  the  whole  domain  of  gunnery, 
or  to  hit-and-miss  aspects  of  the  domain,  they  can  be  evaluated  for 
individually  meaningful  objectives.  Based  on  these  evaluations,  recommen¬ 
dations  for  device  use  can  be  targeted  at  specific  units  of  instruction. 
The  effect  of  this  is  to  keep  recommendations  for  device  use  subordinate 
to  considerations  of  training  structure  and  sequence  as  described  above. 
Also  by  targeting  evaluations  to  objectives,  testing  capabilities  can  be 
identified  for  well-defined  domains  of  performance  which  have  meaning  in 
terms  of  progressions  of  skills  acquisition.  This  jnsures  that  training 
devices  are  put  to  optimum  use  within  an  integrated  program  for  testing 
and  training  gunnery  skills. 


Gunnery  performance  occurs  within  a  man-machine  interactive  system. 
From  the  organizational  perspective,  individual  crew  behavior  is  not  the 
ultimate  criterion;  system  performance  and  winning  engagements  are  the 
criteria  of  most  significance.  System  performance  is  a  product  of  both 
machine  and  persons.  Overall  system  performance  is  less  relevant, 
however,  for  purposes  of  training  and  testing  individual  proficiency. 

That  statement  is  from  a  psychological,  rather  than  organizational 
perspective.  Both  perspectives  are  important,  but  for  different  reasons. 
They  should  not  be  confused.  In  this  research,  little  emphasis  is  placed 
on  "hitting  targets"  as  a  criteria;  the  emphasis  is  on  the  individual 
behaviors  that  are  required  for  the  system  to  hit  targets. 


Measurement  of  Conditions,  Knowledges,  Behaviors,  and  Outcomes 

Our  testing  approach  is  based  on  a  simple  model  of  performance 
relating  conditions  and  cues,  knowledge,  job  behavior,  and  outcomes  of 
that  behavior  (White,  Borman,  Hough,  &  Hoffman,  1986;  Campbell,  Dunnette, 
Lawler,  &  Weick,  1970).  As  depicted  in  Figure  4-3,  job  behavior  is  a 
product  of  individual  knowledge  applied  to  a  set  of  environmental  condi¬ 
tions.  Performance  consequences  are  determined  by  the  appropriateness  of 
the  behavior  for  the  environmental  conditions.  Evaluation  of  performance 
ultimately  rests  in  the  value  or  utility  of  the  consequences  that  result 
from  any  job  behavior.  Alternative  job  behaviors  are  differentially 
valued  by  the  utility  of  the  various  performance  consequences  they 
produce. 


Figure  4-3.  Performance  model. 


In  simple  procedural  tasks,  like  most  of  tank  gunnery,  the 
relationship  between  behaviors  and  performance  consequences--for  any  given 
environmental  circumstances--are  known.  For  performance  assessment,  this 
means  that  it  is  not  necessary  to  wait  for  the  outcome  in  order  to 
evaluate  the  behavior.  The  values  of  alternative  behaviors  are 
predefined.  For  other  performance  domains  where  the  relationship  between 
behavior  and  job  consequences  is  less  certain,  the  value  of  any  particular 
behavior  can  only  be  determined  after  the  consequences  are  observed. 

The  intent  of  explicating  this  simple  model  is  to  avoid  some  of  the 
confusion  discussed  by  Boldovici  (1979)  concerning  process  from  product 
measurement  for  gunnery  performance.  Our  model  parallels  his  distinction 
of  process  as  means  and  product  as  end,  however  we  choose  to  talk  in  terms 
of  job  behaviors  instead  of  process,  and  job  consequences  or  outcomes 
instead  of  product. 

One  example  of  the  confusion  described  by  Boldovici  (1979)  is  whether 
assessment  of  a  gunner's  sight  picture  is  a  process  or  a  product  measure. 
Using  the  behavior/outcome  distinction,  a  record  of  a  sight  picture  is  not 
a  direct  record  of  a  person's  behavior.  Rather  it  is  a  record  of  the 
consequences  of  a  person's  behavior.  Less  relevant  for  the  analysis  of 
tank  gunnery,  but  an  example  used  by  Boldovici  (1979)  is  the  assessment  of 
a  diver.  His  argument  is  that  the  "product"  of  the  diving  is  a  splash  in 
the  water,  a  difficult  entity  to  evaluate.  Thinking  in  terms  of 
consequences,  we  would  argue  that  the  intended  outcome  ib  the  aesthetic 
pleasure  of  the  spectators.  Scoring  of  diving  competition  is  a  judgment 
of  behavior;  however,  the  scoring  differentially  values  dives  in  terms  of 
the  forms  and  techniques  that  novice  and  seasoned  spectators  can 
appreciate.  Thinking  in  terms  of  consequences  rather  than  products 
broadens  our  perspective  on  job  performance  in  other  ways  as  well. 
Boldovici  (1979)  expresses  some  dissonance  with  the  notion  that  a 
chambered  round,  the  result  of  loading,  is  a  "product."  Again,  it  one 
thinks  in  terms  of  consequences,  a  loaded  round  makes  perfect  sense  as  the 
consequence  of  loading. 

Given  that  our  primary  purpose  for  testing  individual  performance  is 
to  provide  information  for  training  and  skills  acquisition,  the  unit  of 
analysis  we  are  most  interested  in  is  individual  job  behavior.  Individual 
behavior  represents  the  only  way  a  crewman  can  exert  his  influence  and 
perform  his  role  in  the  tank,  whether  the  immediate  relationship  is  with 
other  crew  members,  platoon  leaders,  or  the  tank  itself.  Behavior  is  what 
the  individual  can  and  must  change  in  order  to  affect  crew  performance 
outcomes.  Alternative  levels  of  analysis  include  assessment  of  knowledge 
requirements  for  appropriate  behavior,  and  assessment  of  the  consequences 
of  behavior.  Assessment  of  consequences  and  assessment  of  knowledge  can 
and  should  augment  the  assessment  of  behavior  but  only  under  certain 
c i rcumstances . 

There  are  many  simple  behaviors  that  are  presumed  to  be  part  of  all 
soldiers'  repertoire,  e.g.,  flipping  a  toggle  switch.  If  a  soldier  has 
the  knowledge  that  the  behavior  is  required,  it  is  safe  to  assume  the 
behavior  can  be  executed.  In  fact,  instructions  like  "turn  the  switch  on" 


that  actually  refer  to  the  consequences  are  given  as  instructions  about 
behavior.  In  other  cases,  this  assumption  is  unreasonable.  That  is,  a 
person  may  know  that  a  tennis  ball  is  to  be  served  within  designated 
boundary  lines,  but  that  does  not  mean  that  the  person  can  execute  the 
correct  behavior.  That  is  not  to  say  that  knowledge  of  where  to  serve  the 
tennis  ball  is  not  important.  In  fact,  if  an  athletically  adept  person 
does  not  know  which  of  the  court  lines  define  the  serving  area,  he/she 
will  serve  it  in  the  right  place  only  by  accident.  Our  theoretical  model 
specifies  that  knowledge  is  always  a  necessary,  but  not  always  a  suffi¬ 
cient  condition  for  appropriate  behavior.  If  it  is  sufficient,  then 
assessment  of  that  knowledge  may  be  substituted  for  testing  the  behavior. 
On  the  other  hand,  if  the  behavior  is  complex,  requiring  practice  (a  skill 
as  defined  in  Adams,  1987),  then  assessing  only  the  supoorting  knowledge 
is  insufficient. 

For  some  behaviors,  the  assessment  of  performance  consequences  may 
provide  an  acceptable  index  of  job  behavior.  Two  conditions  jointly 
influence  the  assessment:  (a)  the  complexity  of  the  behavior,  and  (b)  the 
strength  of  the  relationship  between  the  behavior  and  the  consequences  of 
interest.  In  some  cases,  the  behaviors  are  simple  and  outcomes  are 
predictably  related  to  behaviors.  Given  that  a  switch  is  "off"  and  should 
be  "on,"  assessing  the  position  of  the  switch  after  performance  may 
reasonably  be  accepted  as  an  index  of  whether  the  appropriate  behavior 
occurred  or  not.  If  there  are  no  automated  returns  or  positioning  wired 
into  the  switch,  there  is  no  contamination  in  the  measure.  Furthermore, 
failure  to  perform  the  procedure  may  be  corrected  by  instructions  related 
to  the  outcome,  e.g.,  tell  the  person  to  "turn  the  switch  on." 

The  tennis  example  provides  a  somewhat  different  illustration.  The 
behavior  is  not  simple.  It  is  not  easily  described  or  easily  recorded  for 
evaluation.  Furthermore,  there  are  undoubtedly  variations  in  technique 
that  all  result  in  the  ball  landing  within  the  serving  area.  On  the  other 
hand,  barring  poor  equipment  or  weather  conditions,  the  trajectory  and 
landing  of  the  ball  is  a  direct  consequence  of  the  behavior.  Therefore, 
whether  the  ball  is  in-bounds  can  provide  an  evaluation  of  the  satisfac¬ 
toriness  of  serving  technique  (ignoring  for  the  moment  speed  and  spin  of 
the  ball).  But  because  the  behavior  is  not  simple,  providing  instructive 
feedback  can  be  a  problem.  Unlike  the  switch  example,  simply  repeating 
the  desired  outcomes  does  not  tell  us  how  to  change  our  behavior  (assuming 
we  have  been  trying  to  get  the  ball  in  the  serving  area).  Thus,  the 
location  of  the  ball's  impact  may  be  an  acceptable  proficiency  measure, 
but  it  is  not  a  very  good  diagnostic  measure.  More  direct  behavior 
observation  is  needed  to  provide  corrective  feedback. 

Mitring  a  baseball  is  another  example  of  a  complex  behavior,  but  this 
time  the  consequence  is  not  as  directly  connected  to  the  behavior  for 
three  reasons.  First,  there  are  a  variety  of  environmental  conditions 
that  also  influence  performance  consequences.  That  is,  the  hitter  may  or 
may  not  have  assessed  the  situation  correctly  (i.e.  fast  ball  versus  curve 
or  off-speed  pitch).  Perfect  execution  of  the  behavior  for  hitting  a  fast 
ball  will  result  in  a  miss  at  a  slower  off-speed  pitch.  Or,  if  contact  is 
made,  where  the  ball  is  hit  combined  with  the  ability  of  the  fielders  will 


influence  whether  or  not  the  batter  arrives  safely  on  base.  Thus,  while 
batting  average  is  an  accepted  index  of  hitting  proficiency,  it  ignores 
potential  contamination  due  to  the  abilities  of  opposing  pitchers  and 
fielders.  That  is,  a  hitter  should  want  to  be  on  the  team  with  the 
strongest  pitchers  not  only  because  their  chances  of  winning  are 
increased,  but  also  because  they  won't  have  to  face  those  pitches  and 
their  batting  averages  will  be  higher.  Second,  missing  a  pitch  may  be  due 
to  poor  batting  technique  or  poor  judgement.  Thus,  in  this  example, 
getting  a  hit  or  making  an  out  is  a  contaminated  measure  of  proficiency, 
and  behavior  observation  alone  may  lead  to  ambiguity  in  diagnosing  the 
cause  of  an  errant  swing,  e.g.,  should  instructions  be  behavioral  (let  up 
on  the  swing)  or  cognitive  (watch  for  the  slow  pitch).  Third,  within 
limits,  there  are  a  variety  of  ways  to  hit  a  baseball.  Some  batters 
choke-up,  some  don't;  some  use  an  open  stance,  others  a  closed  stance. 
Variations  in  specific  behavior  techniques  are  used  by  different  hitters 
depending  on  what  consistently  gives  them  the  most  solid  contact  with  the 
ball.  This  suggests  that  successful  measurement  plan  will  include 
recording  of  conditions  (what  pitch  was  thrown),  hitter's  knowledge 
(discriminating  fast  balls  from  off-speed  pitches),  observing  the  swing, 
observing  where  and  how  hard  the  ball  was  hit,  and  observing  whether  or 
not  the  hitter  made  it  to  base.  While  the  later  event  is  the  most 
important  for  the  outcome  of  the  game,  it  is  the  least  important 
assessment  for  instructing  the  batter.  Furthermore,  no  one  measure  alone 
is  sufficient.  Useful  feedback  must  be  based  on  inferences  about  hitters' 
ability  to  judge  pitches,  their  swing,  and  how  well  they  make  contact. 

Thus,  while  behavior  is  the  unit  of  analysis  that  is  of  most 
interest,  it  may  or  may  not  be  the  unit  of  analysis  on  which  a  measurement 
system  should  focus.  Other  things  being  equal,  direct  assessment  of 
behavior  is  preferred.  But  that  is  not  always  so.  In  some  cases 
knowledge  assessment  may  be  easier  to  acquire  and  may  suffice  as  an 
appropriate  substitute.  In  other  cases,  it  may  be  required,  or  not 
necessary.  Likewise,  in  some  cases  assessment  of  outcomes  or  consequences 
may  suffice,  whereas  in  others  it  is  too  contaminated  or  too  difficult  to 
obtain.  Outcome  measures  are  seductive  in  that  they  often  tap  the  level 
of  analysis  of  most  interest  to  system  designers  and  planners.  For 
example,  it  takes  "steel  on  target"  to  win  battles,  and  tacticians  must 
plan  according  to  steel  on  target  capability.  For  individual  or  crew 
level  assessment,  steel  on  target  is  contaminated  by  system  errors  and 
measurement  difficulties.  More  importantly,  it  does  not  yield  the  kind  of 
diagnostic  information  needed  to  provide  corrective  feedback. 


Measurement  Classification  Rules 

The  principles  underlying  the  above  discussion  are  not  hard  and  fast 
rules.  For  example,  it  is  a  subjective  judgment  whether  an  outcome  is  the 
result  of  too  many  factors  other  than  the  behavior  of  the  individual  to  be 
a  useful  focus  of  measurement  of  individual  performance.  However,  the 
principles  do  provide  useful  heuristics  for  specifying  the  kinds  of 
measures  most  likely  to  be  useful  in  the  training  context.  That  is,  it  is 
possible  to  analyze  the  performance  elements  in  the  gunnery  domain  and 


specify  the  kind  of  measure  or  measures  that  seem  best  suited  to  measuring 
proficiency  and  providing  feedback.  Table  4-1  provides  a  summary  of  these 
rules. 


Table  4-1 

Factors  Affecting  Measurement  Mode  Selection  for  Individual  Assessment 


1.  Cognitive  requirement: 

a.  Linear  procedure,  simple  reaction.  Feedback  about  knowledge  is 
trivial . 

b.  Choice  reaction  and  behavior  adjustment  required.  Assessment  of 
the  appropriateness  of  choice  is  needed  to  diagnose  incorrect 
performance. 

2.  Behavior  requirement: 

a.  Simple.  Feedback  about  behavior  per  se  is  trivial. 

b.  Complex.  Direct  assessment  of  behavior  is  needed  to  diagnose 
incorrect  performance. 

3.  Consequences: 

a.  Strong  association  with  behavior;  outcomes  primarily  dependent  on 
behavior.  Outcome  measures  may  be  sufficient  for  feedback  if  the 
behavior  is  simple. 

b.  Weak  association;  outcomes  heavily  dependent  on  factors  other 
than  individual's  behavior.  Outcome  measures  are  too  contaminated 
to  be  certain  about  the  behavior  that  took  place. 

c.  Alternative  associations.  There  are  alternative  behaviors  for 
obtaining  desired  outcomes.  Can  not  infer  what  behavior  took  place 
from  observing  outcomes. 


There  are  twelve  combinations  of  the  three  factors  affecting 
measurement  mode.  However,  for  the  combinations  that  involve  alternative 
behaviors,  both  simple  and  complex  behavior  types  require  observation  of 
the  behavior  to  ascertain  what  was  done.  Thus,  no  distinction  in  type  was 
made  between  simple  and  complex  behavior  when  alternative  behavior 
associations  with  outcomes  could  occur.  As  a  result  there  are  ten  types 
of  elements.  The  types  and  their  measurement  specification  rules  are 
identified  in  Table  4-2. 

Two  additional  classifications  of  performance  elements  are  added  to 
the  ten  to  cover  task  elements  that  involve  no  overt  behavior  and  no 
immediately  observable  consequences.  These  cases  still  fit  the  model;  the 
difference  is  simply  that  neither  the  behavior  nor  the  consequence  is 


Table  4-2 


Measurement  Specification  Rules 


Step 

Type 

Decision 

Behavior 

Consequences 

Measurement  Requirement3 

Overt 

Behaviors: 

1. 

No 

Simple 

Strong  Assoc. 

K, 

B  or  0  will  suffice 

2. 

No 

Simple 

Weak  Assoc. 

K 

or  B  will  suffice 

3. 

No 

Complex 

Strong  Assoc. 

B 

necessary; 

0  for  proficiency  only 

4. 

No 

Complex 

Weak  Assoc. 

B 

necessary 

5. 

No 

Either 

Alternative 

B 

and  0  necessary 

Associations 

6. 

Yes 

Simple 

Strong  Assoc. 

K 

necessary; 

B  or  0  for  proficiency 

7. 

Yes 

Simple 

Weak  Assoc. 

K 

necessary; 

B  for  proficiency 

8. 

Yes 

Complex 

Strong  Assoc. 

K 

and  B  necessary; 

0  for  proficiency 

9. 

Yes 

Complex 

Weak  Assoc. 

K 

and  B  necessary 

10. 

Yes 

Either 

Alternative 

K, 

B  and  0  necessary 

Associations 

Covert 

Behaviors 

: 

11. 

No 

Observing 

Information 

K 

for  new  information 

12. 

Yes 

Analyzing 

Information 

K 

for  analysis  rules  and 

K  for  new  information 

aK  =  Knowledge  assessment;  B  =  Behavior  assessment;  0  =  Outcome  assessment 


ordinarily  observable.  There  are  covert  behaviors  and  individually 
private  consequences.  In  each  case,  covert  observation  or  analysis  of 
environmental  cues  provide  the  individual  with  new  information  for  use  in 
performing  later  task  elements.  These  two  kinds  of  task  elements  could 
have  been  eliminated  by  combining  them  with  the  next  overt  behavior  in  the 
procedure.  However,  in  the  instances  where  such  elements  appear,  the 
requisite  knowledge  or  the  analysis  process  was  judged  sufficiently 
complex  to  warrant  attention  apart  from  the  subsequent  overt  behavior. 

By  categorizing  each  gunnery  element  by  type,  measurement 
requirements  become  implicit.  In  order  to  classify  elements,  the  most 
immediate  knowledge  requirement  and  the  most  immediate  consequence  were 
considered  along  with  the  overt  behavior.  Examples  of  each  category  of 
element  are  presented  in  Table  4-3. 
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Table  4-3 


Measurement  Category  Examples  for  the  Ml  Tactical  Gunnery  Domain 


Step 

Type 

Crew 

Member 

Activity 

Description 

Comments 

1. 

Gunner 

2 

Index  battlecarry  ammo 
using  AMMO  SELECT  switch 

Simple  action;  outcome 
is  switch  position. 

2. 

TC 

4.1 

Issue  Fire  Command 

Outcome  is  crew  response 
which  depends  on  crew  as 
well  as  TC,  therefore 
outcome  association  is 
weak. 

3. 

Gunner 

4.1 

Squeeze  trigger(s) 
(reticle  on  target) 

Outcome  is  sight  picture, 
strongly  association  with 
behavior.  Behavior  is 
complex;  coaching  needs 
to  consider  such  things 
as  flinching,  etc. 

4.  None  of  the 

tactical 

gunnery  elements  were  classified  in  this  category. 

5. 

Loader 

4.1 

Check  turret  ring 

Behavior  is  simple,  but 
potential  outcomes  of 
checking  versus  not 
checking  very.  If  there 
are  no  obstructions, 
there  is  no  outcome 
indicator  of  whether  or 
not  loader  checked  ring. 

6. 

TC 

2.2 

Prepare  tank  sketch  card 

Must  decide  what  to  draw, 
but  sketch  card  is  a 
direct  record  TC's  work. 

7. 

Gunner 

8 

Aid  in  adjusting  TC's 
weapon 

Must  determine  what 
directions  to  give  TC, 
but  TC  may  or  may  not 
follow  those  directions. 

8. 

Driver 

3.1 

Follow  wingman  concept; 
react  to  formation 

Position  of  tank  is 
outcome. 

changes 

(table  continues) 
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Step  Crew 

Type  Member  Activity 


Description 


Comments 


TC 

4.3 

Direct  GNR  onto  target 
using  one  of  the 
following  techniques: 

.  use  verbal  commands: 

.  use  Target  Reference 
point 

.  announce  WATCH  MY 

TRACERS  and  use  CAL  .50 
to  point  to  target 

TC  decides  what  to  and 
must  do  if  correctly, 
but  outcome  (gunner 
identifying  target) 
also  depends  on  gunner. 

TC 

2 

Supervise/assist  in 
boresighting  main  gun 

Requires  decisions 
during  procedure, 
behaviors  are  complex 
and  outcome  (bores ighted 
tank)  depends  on  other 
crew  members. 

Gunner 

3.2 

Estimate  range  to 
evaluate  LRF  return 

Gunner  observes  range 
so  that  he  can  use  it 
later  when  evaluating 

LRF  return. 

All 

3.2 

Identify  target(s) 
making  the  following 
determinations: 

•  IFFN 

•  nomenclature 

Determination  later 
used  for  target 
confirmation. 

Measurement  in  the  Context  of  Gunnery  Activities 


So  far  in  the  discussion,  we  have  addressed  testing  as  if  we  were 
interested  in  only  one  element  of  gunnery  at  a  time.  There  are,  however, 
other  considerations  that  have  to  do  with  the  fact  that  any  one  element  is 
significant  only  because  it  is  part  of  a  string  of  elements  that  make  up 
an  integrated  unit  of  performance.  The  most  important  of  these 
considerations  is  that  testing  should  be  designed  not  around  elements,  but 
by  activities  and  objectives.  That  is,  to  isolate  an  element  and  remove 
it  from  the  context  of  the  rest  of  the  elements  in  the  sequence  to  which 
it  belongs  removes  many  external  and  internal  cues,  e.g.,  kinematic  cues 
related  to  movement  sequences  (Adams,  1987).  According  to  cognitive 
learning  theories  such  as  Anderson  (1982),  those  persons  with  high  levels 
of  skill  do  not  perform  in  element-by-element  fashion,  but  by  higher  order 
units  of  behavior  or  macro  productions  (see  previous  section).  Thus,  test 
administration  should  be  organized  at  the  macro  production  level,  arranged 
following  the  hierarchical  analyses  as  objectives.  Furthermore,  all 
elements  must  be  represented  in  the  test  because  a  production  is  defined 
by  its  individual  elements  and  omitting  elements  defines  a  different 
production. 

Having  introduced  the  concept  of  macro  productions,  it  is  necessary 
to  go  back  and  elaborate  somewhat  on  our  concepts  of  knowledge  versus 
behavior  testing.  In  the  context  of  chunking  and  productions,  we  do  not 
mean  to  imply  that  testing  "knowledge"  requires  a  written  test  of  some 
kind.  Procedural  knowledge,  particularly  if  the  procedure  is  well  learned 
is  not  necessarily  verbal  nor  easily  accessed  by  verbal  means  (Anderson, 
1982).  Our  distinction  between  elements  that  can  be  tested  by  knowledge 
assessment  versus  those  that  require  behavior  assessment  is  not  based  on 
how  the  element  is  coded  in  memory,  but  rather  on  how  easy  it  is  to 
translate  the  element  for  verbal  description  to  behavior.  Elements  that 
we  label  as  appropriate  for  knowledge  testing  are  elements  that  are  easily 
described  (verbally)  and  then  executed.  Thus,  failure  to  execute  such  an 
element  during  performance  of  the  procedure  simply  implies  that  the 
sequence  has  not  been  properly  stored  in  some  part  of  memory,  whether  that 
is  verbal  or  some  other  part.  Acquisition  of  those  elements  is  most 
concerned  with  compiling  the  elements  in  proper  sequence.  Elements  that 
we  coded  as  requiring  behavior  assessment  are  those  that  require  greater 
motor  skill,  and  consequently  more  attention  to  acquiring  correct  motor 
behaviors.  The  two  categories  of  elements  are  not  clearly  discrete.  They 
are,  however,  useful  distinctions  for  guiding  appropriate  selection  of 
feedback  information.  For  "knowledge"  elements,  we  are  concerned  with 
whether  or  not  the  behavior  occurs.  For  "behavior"  elements,  we  have  an 
additional  concern  with  how  well  or  which  behavior  occurred. 


Other  Testing  Considerations 


Other  testing  considerations  introduced  at  the  objectives  level  are 
test  use,  speed  of  performance,  hierarchical  arrangement  of  objectives  and 
activities,  testing  performance  under  different  environmental  conditions, 
and  use  of  composite  scores. 
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Test  Use.  Performance  testing  in  the  training  context  has  two 
potential  uses:  (a)  diagnostic  testing  to  pinpoint  skill  deficiencies,  and 
(b)  proficiency  testing  to  assess  level  of  performance.  Certainly  the  two 
uses  are  not  mutually  exclusive,  but  there  are  two  major  differences 
having  to  do  with  treatment  of  scores  and  use  of  sampling.  It  has  been 
argued  above  that  testing  should  be  organized  around  activities  and  that 
all  elements  should  be  included.  For  diagnostic  testing,  that  requirement 
is  extended:  Each  element  must  be  scored  and  the  scores  be  made 
available.  For  proficiency  estimation  on  the  other  hand,  some  sort  of 
aggregate  or  total  score  (e.g.  percent  of  elements  performed  correctly)  is 
more  appropriate.  Obviously,  a  test  can  provide  both.  However,  the  fact 
that  individual  element  scores  are  not  needed  for  proficiency  testing 
creates  the  possibility  of  sampling  activities  and  inferring  other  scores 
based  on  the  sample  observed.  For  example,  if  the  relationship  between 
single  target  gunnery  and  fire  adjustment  is  sufficiently  strong,  fire 
adjustment  proficiency  could  be  adequately  estimated  from  single  target 
gunnery  test  scores.  However,  specific  deficiencies  in  fire  adjustment 
could  not  be  identified.  Thus,  diagnostic  testing  requires  scoring  and 
feedback  of  all  elements  and  activities,  whereas  proficiency  testing 
allows  for  sampling  of  activities  and  use  of  summary  scores. 

Speed.  Speed  of  performance  is  an  important  metric  for  essentially 
all  of  the  gunnery  activities,  particularly  the  engagement  activities. 
Acquisition  time,  opening  time,  and  hit  time  are  the  times  most  often 
assessed.  Each  of  these  represent  the  time  it  takes  for  the  crew  to 
execute  a  series  of  task  elements.  Times  to  execute  other  elements, 
either  individually  or  grouped,  can  be  imagined.  There  are  limits  to  what 
is  possible  to  time  on  the  tank  with  traditional  techniques,  but  a  scorer 
with  an  electronic  stopwatch  can  record  as  many  events  as  there  are 
memories  in  the  watch.  For  example,  depending  on  his  ability  to  see  and 
hear,  a  scorer  on  a  tank  range  could  capture  times  for  (a)  when  the  TC 
announces  "GUNNER,"  (b)  when  the  gunner  announces  "IDENTIFIED,"  (c)  when 
the  gunner  lases,  (d)  when  the  TC  announces  "FIRE,"  (e)  when  the  gunner 
fires  (f)  when  the  loader  announces  "UP"  for  the  subsequent  round,  (g) 
when  the  gunner  announces  his  observation,  and  (h)  when  the  TC  announces 
his  next  command.  A  difficulty  is  maintaining  a  record  of  the  sequence  of 
events.  Extending  the  idea  to  sophisticated  simulators,  however,  one  can 
imagine  performance  profiles  giving  time  to  execute  each  element.  Such 
profiles  could  be  useful  to  pinpoint  particular  elements  in  the  procedure 
that  seem  to  be  slowing  down  a  crew's  performance. 

There  are  two  caveats  regarding  time  as  a  performance  metric.  First, 
time  by  itself  is  an  insufficient  metric.  It  may  show  what  behaviors  are 
slow,  but  it  does  not  provide  any  information  about  why.  The  second  has 
to  do  with  the  pervasive  speed  and  accuracy  trade-off:  For  many  tasks, 
increased  speed  leads  to  reduced  accuracy,  and  vice  versa.  The  question 
becomes  what  combination  of  speed  and  accuracy  is  best.  That  is,  if 
utility  of  performance  is  viewed  as  a  function  of  speed  and  accuracy,  what 
is  the  function?  In  general,  the  rule  seems  to  be  "the  taster  the  better 
as  long  as  the  behaviors  are  performed  correctly."  For  example,  speed  as 
a  measure  of  boresighting  proficiency  make  sense  only  for  discriminating 
among  crews  who  can  in  fact  boresight  the  Ml. 
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For  those  portions  of  the  gunnery  domain  that  involve  engaging 
targets,  there  has  been  considerable  debate  about  how  time  and  performance 
accuracy  should  be  combined  into  a  performance  utility  measure  (Witmer, 
1986;  0.  W.  Bessemer,  personal  communication,  September  1982).  This 
debate  is  most  relevant  to  the  organizational  perspective  of  assessment 
performance  for  the  purpose  of  analyzing  battle  capabilities  where  the 
trade-off  between  speed  and  accuracy  can  be  evaluated  for  its  impact  on 
mission  success.  Those  discussions  are  less  relevant  for  our  purposes  and 
will  not  be  extended  here.  Instead,  we  are  concerned  with  how  to  use 
speed  and  accuracy  feedback  to  best  facilitate  skill  acquisition. 

According  to  common  "stage"  notions  of  skills  acquisition  (e.g.,  Fitts, 
1964;  Anderson,  1982),  performance  accuracy  precedes  performance  speed  in 
the  skill  acquisition  process.  Furthermore,  instructions,  feedback,  and 
payoffs  for  emphasizing  speed  lead  to  decrements  in  performance  accuracy 
(Fitts,  1966;  Howell  &  Kreidler,  1964).  Thus,  we  will  adopt  the  training 
prescription  that  accurate  performance  should  be  the  initial  emphasis  of 
practice,  with  speed  emphasized  only  after  correct  behaviors  are  acquired. 

Consequently,  we  will  not  make  complex  prescriptions  about  how  to  mix 
accuracy  (from  either  knowledge,  behavior,  or  outcome  sources)  and  time 
measures.  Instead,  time  should  be  used  during  practice  only  when  the  task 
elements  are  performed  correctly. 

Hierarchical  Arrangement  of  Activities.  The  tactical  gunnery  domain 
has  been  described  by  partitioning  it  into  activities.  In  addition,  our 
hierarchical  analysis  delineating  training  objectives  further  divides  the 
domain  within  and  across  activities.  For  complete  diagnostic  testing,  all 
activities  and  all  elements  within  them  must  be  testable  in  order  to  be 
able  to  provide  specific  diagnostic  feedback  about  what  needs  remediation. 

The  hierarchical  arrangement  of  the  activities  of  the  Ml  tactical 
gunnery  domain  (detailed  in  previous  section)  is  based  on  the  notion  that 
there  are  dependencies  among  the  activities.  That  is,  proficiency  on  some 
activities  may  be  prerequisite  or  propadeutic  for  the  performance  of  other 
activities.  This  information  may  be  used  in  testing  applications  to 
shorten  testing  time  through  adaptive  testing  strategies.  That  is,  if  the 
hierarchy  is  valid,  then  testing  key  activities  first  may  provide 
information  about  the  likely  performance  in  other  activities.  For 
example,  if  a  gunner  can  engage  multiple  main  gun  targets  from  a  moving 
tank,  there  may  be  a  very  high  probability  that  he  can  engage  single  main 
gun  targets.  Thus,  testing  performance  with  multiple  targets  first  would 
eliminate  the  need  to  test  single  targets  for  crews  who  are  proficient  at 
multiple  targets.  On  the  other  hand,  given  that  the  domain  of  tactical 
gunnery  is  broad  rather  than  deep  (see  previous  section),  there  may  not  be 
many  opportunities  to  infer  performance  in  one  area  from  that  in  another. 

Testing  samples  of  the  domain  may  be  appropriate  for  less  intensive 
proficiency  level  determination.  Sample  requirements  are  determined  by 
domain  heterogeneity.  The  less  the  covariation  among  all  performance 
elements  in  the  domain,  the  more  elements  must  be  observed  to  obtain 
reliable  measurement. 


In  addition  to  hierarchical  dependencies,  there  may  be  other  reasons 
for  covariation  among  performance  scores  for  the  various  activities 
increasing  the  possibility  of  generalizing  proficiency  from  a  sample  of 
activities  to  the  entire  domain  of  activities.  For  example,  covariation 
may  result  because  many  of  the  activities  tend  to  be  practiced  together. 
Note  that  correlations  among  activity  performance  scores  that  are  the 
spurious  result  of  experience  do  not  support  any  conclusions  about 
transfer  of  learning  between  activities.  The  need  to  specifically  learn 
each  activity  is  maintained  as  well  as  the  need  for  assessing  all  elements 
for  diagnostic  testing.  Sampling  of  activities  is  appropriate  only  when 
measuring  proficiency. 

Domain  Conditions.  Organization  of  the  gunnery  activities  was  based 
on  the  parameters  of  the  gunnery  environment  (see  Chapter  2).  Separate 
activities  were  created  to  reflect  the  impact  that  some  of  the  varied 
parameter  conditions  have  on  gunnery  behavior  requirements.  These 
conditions  were  labeled  primary  conditions.  Also,  a  number  of  secondary 
conditions  were  identified  that  do  not  directly  influence  what  behaviors 
should  be  performed,  but  do  influence  how  and/or  how  well  they  are 
executed.  How  well  performance  under  many  of  these  difference  conditions 
relate  to  each  other  is  unknown. 

For  some  activities,  performance  under  the  different  secondary 
conditions  may  be  ordered  by  varying  degrees  of  difficulty.  For  these 
activities,  it  is  appropriate  to  describe  level  of  proficiency  in  terms  of 
conditions  and  standards.  For  other  activities  and  parameters,  there  may 
be  no  necessary  ordering  by  difficulty,  but  simply  a  correlation.  In  such 
cases,  when  not  all  conditions  can  be  tested,  sampling  is  appropriate. 

The  two  categories  of  parameters  (primary  and  secondary)  have 
different  implications  for  testing.  Recall  that  primary  conditions  are 
those  that  must  be  presented  in  order  to  branch  a  crew  through  each  of  the 
parts  for  the  Ml  tactical  gunnery  domain.  Specification  of  conditions  of 
these  parameters  creates  the  environment  that  a  crew  must  attend  to  in 
tactical  gunnery.  These  may  be  conceived  of  as  essential  foreground 
elements  in  the  gunnery  environment.  In  contrast,  the  secondary 
parameters  are  background  elements  of  the  gunnery  environment  that  may 
influence  performance  to  an  unknown  degree.  The  foreground  conditions 
must  be  represented  in  a  simulation  if  all  of  the  gunnery  activities  and 
options  within  the  activities  are  to  be  covered  in  tactical  gunnery 
training  and  skill  assessment.  Requirements  for  representation  of  the 
background  conditions  are  less  certain,  however  assessing  proficiency 
under  varying  background  conditions  will  increase  the  reliability  of 
measurement  and  therefore  increase  confidence  that  proficiency  scores  can 
be  generalized  to  a  variety  of  circumstances. 

Composite  Scores  and  Outcome  Scores.  We  have  argued  that  for 
diagnostic  testing  feedback  needs  to  be  available  for  every  element  of 
performance.  On  the  other  hand,  summary  information  is  more  convenient 
for  conveying  general  competence.  Thus,  detailed  scores  can  be  aggregated 
into  composite  indices  of  performance  at  various  levels  of  abstraction. 
Because  they  are  an  abstraction,  the  primary  concern  of  a  composite  score 
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is  the  underlying  logic  used  to  turn  a  set  of  scores  about  individual 
elements  of  behavior  into  one  index.  It  is  important  than  that  composites 
be  logically  constructed  with  meaning  that  is  easily  understood  and  is 
appropriate. 

A  composite  score  is  a  single  score  constructed  from  a  number  of 
other  scores  by  a  series  of  rules  or  by  a  mathematical  formula.  The  set 
of  rules  or  the  formula  gives  the  composite  score  its  meaning.  It  is 
useful  to  think  of  that  meaning  as  expressing  the  value  or  utility  of  the 
particular  aggregation  of  behaviors.  Thus,  the  composite  score  can  be 
viewed  as  the  resultant  of  a  utility  function.  But,  utility  for  what? 
There  are  several  kinds  of  decisions  and  purposes  for  composite  scores, 
including  general  training  management  decisions,  personnel  placement, 
mission  planning  and  analysis,  crew  motivation,  and  training  progress 
indices. 

Two  different  kinds  of  utility  functions  may  be  identified  in 
relation  to  these  uses.  The  first  is  a  derivative  of  the  performance 
model  presented  in  Chapter  4.  That  is,  composite  scores  may  express  the 
utility  of  a  given  set  of  behaviors  in  terms  of  the  value  of  the  expected 
consequences  or  outcomes  of  those  behaviors.®  The  nature  of  a  composite 
is  such  that  there  may  be  trade-offs;  that  is,  equal  functional  value  may 
be  assigned  to  more  than  one  pattern  of  behavior.  The  formula  of  con¬ 
structing  the  composite  score  should  equate  those  alternative  patterns. 
There  may  be  trade-offs  (at  least  for  some  objectives)  such  that  faster 
but  somewhat  less  accurate  behaviors  will  lead  to  as  much  "productivity" 
as  slow  but  more  accurate  behavior.  Hit  rate  (controlling  for  equipment 
and  ammunition  errors)  is  a  product  of  accurate  behaviors  (including 
everything  from  target  acquisition  to  fire  adjustment)  and  speed  of 
executing  those  behaviors.  Within  limits,  there  are  trade-offs  between 
speed  and  accuracy  (expressed  in  a  non-linear  ration  function  such  as  hit 
per  time  or  time  to  hit)  resulting  in  equivalent  utility.  Thus,  one  type 
of  composite  may  be  constructed  to  represents  crews'  current  level  of 
proficiency  in  the  real  world  setting  according  to  the  expected 
consequences  of  their  behavior. 

The  second  kind  of  utility  function  expresses  performance  in  terms  of 
training  value  or  expected  increment  in  performance  as  a  result  of  a 
practice  trial.  In  a  sense,  this  is  a  derivative  of  the  first  option,  but 
it  shifts  attention  from  a  real  world  focus  to  a  training  focus.  One 
reason  for  making  the  distinction  evolves  from  the  previously  cited 
research  concerning  the  speed  versus  accuracy  trade-off.  We  have  argued 
that  when  practicing  for  performance  improvement,  speed  should  not  be 

^We  use  the  term  "expected  consequences"  in  recognition  of  the  fact 
that  performance  consequences  are  a  product  of  crew  behaviors  plus 
equipment  functioning.  Thus,  "correct"  behaviors  may  not  always  lead  to 
desired  consequences.  However,  from  the  standpoint  of  the  crew 
instruction  and  evaluation,  the  behavior  was  no  less  correct.  Similarly, 
incorrect  behaviors  may,  by  chance,  lead  to  desired  outcomes,  but  that 
does  not  make  those  behaviors  "correct." 


emphasized  until  procedural  accuracy  is  achieved.  Thus,  for  performance 
improvement,  the  utility  function  may  not  incorporate  the  same  speed  and 
accuracy  tradeoff  as  the  proficiency  function.  In  the  early  stages  of 
learning,  the  training  payoff  for  any  practice  trial  may  have  little  to  do 
with  speed,  and  thus  speed  should  not  be  incorporated  in  the  composite 
(e.g.,  receive  zero  weight  in  a  linear  utility  function).  Later  in 
training,  as  some  degree  of  accuracy  increases,  adding  speed  during 
practice  is  necessary  and  should  be  included  in  a  composite.  At  present, 
not  enough  is  known  about  the  acquisition  process  to  make  specific 
recommendations  concerning  when  to  begin  emphasizing  speed. 

There  may  be  other  such  trade-offs  in  constructing  composite  scores. 
The  important  point  is  that  a  composite  score  should  be  constructed  for  a 
specific  purpose.  Two  purposes  have  been  explicated:  (a)  evaluating  the 
utility  of  performance  in  terms  of  expected  consequences,  and  (b)  evaluat¬ 
ing  practice  in  terms  of  skills  acquisition.  Composite  scores  that 
express  performance  in  terms  of  real  world  utility  support  general 
training  management  decisions  (e.g.,  are  crews  at  desired  levels  of 
performance  or  should  more  time  be  allocated  for  training?)  and  mission 
planning  and  analysis  (e.g.,  given  current  proficiency  how  much  fire  power 
can  be  expected?).  Furthermore,  they  can  be  used  to  influence  general 
levels  of  crew  motivation  by  appropriate  connections  with  extrinsic 
(awards,  time-off)  and  intrinsic  (self-competence,  esprit  de  corps) 
reinforcements.  Composites  constructed  to  represent  utility  for  improved 
performance  made  be  used  for  more  specific  training  progress  determina¬ 
tions  and  can  be  used  to  reinforce  specific  response  patterns  during 
training.  Construction  of  an  equation  for  a  proficiency  composite  is  a 
relatively  straightforward  operations  research  problem.  Construction  of  a 
utility  equation  reflecting  training  value  of  a  practice  trial  is  a 
psychological  problem  for  which  a  solution  is  not  readily  available 
because  research  on  gunnery  training  has  focused  on  transfer  issues  rather 
than  on  patterns  of  skill  acquisition. 


Conclusions 


A  number  of  basic  testing  issues  for  the  Ml  tactical  gunnery  domain 
have  been  discussed.  These  included  the  assignment  of  knowledge,  behavior 
or  outcomes  assessments  for  every  element  as  appropriate  for  diagnostic  or 
proficiency  test  use,  with  testing  of  elements  embedded  in  testing  of 
activities.  Scoring  of  all  elements  should  be  available  for  diagnostic 
use,  however  selective  scoring  may  be  possible  to  estimate  proficiency 
level.  Selective  testing  may  be  guided  by  activity  hierarchies  when 
prerequisite  or  propadeutic  relationship  are  involved,  otherwise,  sampling 
may  be  used.  In  addition,  secondary  conditions  may  be  used  to  define 
levels  of  difficulty  within  affected  activities.  Finally,  composite 
scores  may  be  constructed  to  provide  summary  information  on  performance, 
if  careful  attention  is  given  to  the  meaning  of  composite  scores. 
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CHAPTER  5 


IDENTIFICATION  AND  EVALUATION  OF  DEVICE  FEATURES 
REQUIRED  FOR  TRAINING 


The  purpose  of  the  present  chapter  is  to  evaluate  the  four  gunnery 
training  devices  that  were  designated  for  study:  VIGS,  TopGun,  U-COFT, 
and  SIMNET.  The  chapter  provides  an  exarination  of  device  features  in 
order  to  evaluate  their  utility  for  the  proposed  training/testing  program. 
Training  simulator  features  are  commonly  divided  into  two  general  classes: 
fidelity  features  and  instructional  features.  Fidelity  features  are 
defined  as  those  simulator  components  that  enable  the  simulator  to  mimic 
the  operational  equipment.  In  contrast,  instructional  features  are  those 
simulator  capabilities  that  facilitate  the  instructional  process. 
Accordingly,  the  present  chapter  is  divided  into  two  major  sections  that 
correspond  to  this  distinction.  Before  getting  into  those  topics  we  begin 
with  an  introductory  section  that  describes  the  four  devices  and  the 
procedures  used  to  evaluate  them. 


Evaluation  Methods 


Two  disclaimers  should  be  stated  at  the  outset  of  the  description  of 
our  evaluation  methods.  First,  the  purpose  of  our  evaluation  was  not  to 
mathematically  determine  the  "optimal"  or  "best"  device  for  training 
gunnery.  To  make  such  a  determination  would  require  quantitative  informa¬ 
tion  about  device  effectiveness  and  costs  that  is  neither  available  nor 
easily  obtained.  [See  Sticha,  Blacksten,  Knerr,  Morrison,  and  Cross 
(1986)  for  a  detailed  discussion  of  the  rationale  and  requirements  for 
such  an  analysis.]  Rather,  our  purpose  was  to  compare  the  relative 
strengths  an.  weaknesses  of  specific  gunnery  devices  for  training  the 
domain  of  gunnery  as  defined  in  previous  chapters.  Thus,  a  qualitative 
comparison  of  devices  was  deemed  appropriate  for  the  evaluation  of 
training  features  described  in  the  present  chapter  as  well  as  for  the 
evaluation  of  testing  features  described  in  the  next  chapter.  Second,  the 
findings  were  based  on  information  that  was  available  at  the  time  of  the 
evaluation  (April  -  August  1987).  Some  of  the  conclusions  may  not  hold 
because  of  device  software  updates  and  design  changes  that  have  occurred 
since  that  time  period. 


Devices 


The  four  designated  training  devices  are  all  computer-based  systems 
designed  to  train  Ml-related  skills.  Except  for  SIMNET,  each  is  designed 
to  train  gunnery.  The  latter  is  primarily  designed  to  train  platoon 
through  battalion  level  tactics.  Nevertheless,  SIMNET  simulates  many  of 
the  conditions  and  actions  related  to  armor  gunnery.  Each  of  the  devices 
is  described  in  more  detail  below. 


VI6S.  The  Videodisc  Integrated  Gunnery  Simulator  (VIGS)  is  a 
training  device  that  uses  "intelligent"  videodisc  technology,  i.e., 
videodisc  target  images  are  controlled  by  a  microcomputer.  Designed  to 
sit  on  a  table  top,  the  VIGS  gunner's  station  includes  the  gunner's 
primary  sight  (GPS),  power  control  handles,  and  some  of  the  controls  and 
switches  that  appear  in  the  tank.  The  Ml  version  of  VIGS  is  currently 
under  development  by  the  Educational  Computer  Corporation  (ECC)  and  is 
scheduled  for  fielding  in  FY88.  The  version  of  VIGS  that  was  examined  was 
the  prototype  Ml  VIGS  that  was  designed  by  Perceptronics  (MK-1). 

TopGun.  Developed  as  a  successor  to  Battlesight  (an  M60A1  gunnery 
trainer),  TopGun  was  developed  jointly  by  ARI  and  DARPA  as  a  research 
device  to  investigate  the  use  of  an  inexpensive,  arcade-type  video  game 
for  acquiring  and  sustaining  tank  gunnery  skills.  TopGun  is  similar  to 
VIGS  in  that  it  focuses  only  on  gunner  duties.  Approximately  25  Ml 
version  TopGun  devices  will  be  delivered  for  evaluation  in  FY88. 

U-COFT.  The  Unit  Conduct-of-Fire  Trainer  (U-COFT)  is  a  high 
fidelity,  computer-based  gunnery  simulator  for  training  tank  commanders 
and  gunners.  The  major  components  of  the  U-COFT  include  a  crew 
compartment  that  is  a  close  simulation  of  gunner  and  tank  commander 
stations,  an  instructor-operator  station  (IOS)  where  the  trainer  can 
control  training  events,  and  a  general  purpose  computer  that  supports  both 
image  generation  and  training  management  functions.  In  descending  order 
of  priority,  the  four  purposes  of  U-COFT  are  (a)  to  sustain  year-round 
gunnery  performance,  (b)  to  cross-train  loaders  and  drivers  in  gunner 
duties  and  gunners  in  tank  commander  duties,  (c)  to  transition  train  armor 
personnel  to  the  Ml  or  M1A1  tank,  and  (d)  to  provide  nonarmor  personnel 
(cooks,  mechanics,  etc.)  basic  gunnery  training  to  serve  as  battlefield 
replacements  (Griffin  &  Kuma,  1985).  In  contrast  to  the  other  three 
devices,  the  U-COFT  is  presently  in  the  inventory  of  Army  training 
devices.  Fielding  plans  call  for  U-COFT  to  be  sent  to  every  armor 
battalion  and  cavalry  squadron  and  to  be  integrated  into  their  unit 
training  programs. 

SIMNET.  SIMNET  (Simulated  Networking)  is  a  large  scale  research 
project  in  interactive  networked  simulation  sponsored  by  DARPA  and  TRADOC. 
SIMNET  provides  a  simulation  of  the  battlefield  allowing  armor  crews  to 
participate  in  platoon,  company,  and  battalion-level  force-on-force 
engagements.  A  SIMNET  station  consists  of  a  driver's  compartment  and  a 
crew  compartment  with  gunner,  loader,  and  commander  stations.  Thus, 

SIMNET  is  the  only  device  of  the  four  that  simulates  all  four  tank  crew 
positions. 


Procedures  and  Materials 

In  order  to  obtain  an  accurate  impression  of  device  capabilities  and 
limitations,  we  attempted  to  examine  and  operate  the  four  armor  training 
devices  under  consideration  (VIGS,  TopGun,  U-COFT,  and  SIMNET).  Unfor¬ 
tunately,  VIGS  and  TopGun  are  not  yet  available  in  Ml  form.  For  TopGun, 
we  examined  the  predecessor  device  (Battlesight),  which  is  designed  to 


simulate  M60A1  tank  gunnery.  In  addition,  we  were  furnished  design 
specifications  for  the  Ml  version  of  TopGun.  For  VIGS,  we  examined  the 
prototype  Ml  VIGS  that  was  produced  by  Perceptronics  (MK-1).  Two  examples 
of  VIGS  courseware  were  examined:  (a)  demonstration  courseware  produced 
by  Perceptronics  showing  a  variety  of  friendly  and  threat  targets  under 
day  and  thermal  channel  viewing  conditions,  and  (b)  experimental 
courseware  produced  by  ARI  showing  a  variety  of  thermal  targets.  The 
production  model  of  the  Ml  VIGS  is  currently  under  development  by  the 
Educational  Computing  Corporation.  Design  specifications  for  the  produc¬ 
tion  model  were  not  available  at  the  time  of  evaluation. 


Fidelity  Features 


For  transfer  of  training  to  occur,  we  assume  that  there  must  be  some 
sort  of  correspondence  between  training  device  and  operational  equipment. 
This  assumption  agrees  with  common  sense  notions  of  training  as  well  as 
with  basic  theoretical  concepts  of  transfer.  As  early  as  1903,  Thorndike 
espoused  the  theory  of  identical  elements  to  explain  transfer  phenomena. 

He  maintained  that  the  amount  of  transfer  depends  on  the  number  of 
elements  in  the  training  task  that  are  identical  to  those  in  the  transfer 
task.  Identity  was  defined  in  terms  of  functional  as  well  as  physical 
similarity  between  the  two  tasks.  Similar  transfer  principles  are 
implicit  in  models  that  the  Army  has  developed  to  predict  the  effec¬ 
tiveness  of  training  devices.  (For  a  review  these  models,  see  Tufano  & 
Evans,  1982.)  These  models,  known  collectively  as  TRAINVICE,  all  provide 
for  an  analysis  of  the  physical  and  functional  similarity  between  the 
training  device  and  the  operational  equipment  and/or  an  analysis  of  the 
skills  that  are  common  to  performance  on  the  device  and  on  the  equipment. 
The  results  of  these  analyses  of  devices  are  used  to  predict  the  amount  of 
transfer  that  will  occur  to  the  operational  equipment. 

Similarly,  we  assume  that  the  validity  of  training  devices  as  media 
for  measuring  job  performance  is  dependent  on  the  fidelity  of  the  device 
with  respect  to  the  operational  equipment.  This  assumption  is  based  on 
the  principle  of  content  validity  (Cronbach,  1960).  A  content  valid  test 
is  one  that  provides  a  representative  sample  of  the  to-be-measured 
construct.  Therefore,  the  best  predictor  of  job  performance  would  be  a 
test  that  includes  elements  of  the  job  itself--the  concept  of  the  job 
sample  (Wernimont  &  Campbell,  1968).  Using  this  reasoning,  Black  and 
Graham  (1986)  argued  that  armor  simulators  function  not  only  as  training 
devices,  but  also  provide  job  samples  for  predicting  the  performance  of 
armor  crewmen.  The  relation  of  device  fidelity  to  performance  measurement 
is  discussed  more  fully  in  Chapter  6.  Nevertheless,  our  evaluation  of 
device  fidelity  must  be  mindful  of  the  fidelity  requirements  for  testing 
as  well  as  training. 

The  devices  analyzed  in  the  present  chapter  were  all  designed  to 
simulate  operation  of  the  Ml  tank.  As  a  result  of  these  deliberate 
similarities,  there  are  many  device  characteristics  that  promote  positive 
transfer  to  the  Ml  and  positive  correlations  between  the  performance  on 
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the  device  and  performance  on  the  actual  equipment.  These  device  charac¬ 
teristics  may  be  termed  positive  transfer  and  positive  correlations 
sources,  respectively,  in  that  they  tend  to  increase  transfer  and  correla¬ 
tion  coefficients.  Despite  a  lack  of  empirical  support  for  positive 
transfer  and  correlation  (see  Chapter  3  for  review),  the  devices  have 
innumerable  sources  for  positive  transfer  and  correlation. 

One  reason  for  the  lack  of  empirical  positive  transfer  and  correla¬ 
tion  coefficients  is  that  there  also  exist  specific  dissimilarities 
between  the  device  and  the  operational  equipment.  These  dissimilarities 
may  provide  problems  that  could  constrain  or  possibly  contraindicate  the 
use  of  the  device  for  training  and  testing  certain  aspects  of  gunnery 
(Boldovici,  in  press).  For  instance,  certain  device  characteristics  could 
lead  to  behaviors  that  are  unsafe  on  the  operational  equipment.  Analogous 
to  these  transfer  problems  are  device  characteristics  that  penalize 
appropriate  behavior.  These  latter  characteristics  may  restrict  or 
constrain  the  use  of  training  devices  as  performance  measurement 
instruments.  These  dissimilarities  are  referred  to  as  negative  transfer 
and  correlation  sources  in  that  they  tend  to  reduce  or  attenuate  transfer 
and  correlation  coefficients.  Because  of  their  crucial  impact  on  device 
applications,  these  negative  sources  are  of  particular  concern  in  the 
present  evaluation  and  are  operationalized  as  follows: 

1.  A  negative  transfer  source  is  defined  as  a  training  device 
characteristic  that  results  in  a  novice  learning  an  inappropriate 
(particularly  unsafe)  habit  that  could  transfer  to  performance  on  the 
operational  equipment. 

2.  A  negative  correlation  source  is  a  training  device  characteristic 
that  results  in  an  expert  being  penalized  for  behaving  in  a  manner  that 
would  be  appropriate  in  the  operational  equipment. 


Evaluation  of  Device  Fidelity 

The  results  of  the  evaluation  are  divided  into  two  sections:  (a)  a 
description  of  the  general  capabilities  and  limitations  of  devices,  and 
(b)  a  discussion  of  the  more  specific  sources  of  negative  transfer  and 
correlations. 

General  Capabilities  and  Limitations.  To  describe  the  more  general 
characteristics  of  devices,  fidelity  was  examined  in  terms  of  the  two 
primary  dimensions  of  the  gunnery  domain:  conditions  and  actions.  For 
the  conditions  dimension,  we  determined  whether  devices  could  simulate 
each  condition  within  the  parameters  described  in  Chapter  2.  The  detailed 
results  (in  Appendix  F)  consist  of  a  table  of  "YES"  and  "NO"  entries 
corresponding  to  instances  where  the  device  could  or  could  not  simulate  a 
particular  condition.  These  detailed  results  are  summarized  in 
ii-SAure  5-1,  where  symbols  are  used  to  summarize  the  extent  to  which  each 
gunnery  parameter  is  simulated  by  devices.  Each  rating  was  based  on  the 
proportion  of  conditions  within  parameters  that  are  simulated.  The 


PARAMETERS 


Target  Type 


Target  Movement 


Target  Cover/Conceolment 


Target  Array 


Target  Orientation 


Target  Range 


Target  Sector 


VIGS  TopGun  U-COFT  SIMNET 


Enemy  Activity 


NBC  Conditions 


Equipment  Status 


Number  of  Crewmen 


Supply  Shortages 


Mission 


Fire  Control 


Movement  Formation 


Special  Engagement  Requirements 


ce 


Visibility 


Terrain 


Terrain  Vegetation 
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O 

Q 

O 

All  conditions  (100&)  are  simulated  by  the  device. 

^  Most  conditions  (^50X)  are  simulated  by  the  device. 

Q  Some  conditions  (<  50£)  are  simulated  by  the  device. 

No  Symbol  No  condition  or  only  the  basic  or  null  condition  is  simulated  by  the  device 
Figure  5-1.  Summary  of  conditions  simulated  by  gunnery  training  devices. 


arbitrary  breakpoints  of  100,  50,  and  0  percent  used  as  rating  were 
sufficient  to  describe  the  similarities  and  differences  among  devices. 

Similarly,  the  extent  to  which  every  action  identified  in  the  domain 
could  be  performed  on  the  devices  was  determined  by  answering  some 
"YES/NO"  questions.  The  questions  were  all  phrased  such  that  "NO" 
responses  required  comments  whereas  "YES"  responses  did  not.  The  ques¬ 
tions  were  addressed  in  the  following  order:  (a)  Can  the  action  be 
performed  or  practiced  on  device?  (If  not,  do  not  answer  any  more 
questions.)  (b)  Can  every  subcomponent  (step)  of  the  action  be  performed 
or  practiced?  (c)  Are  the  stimuli  and  responses  equivalent  to  those  on 
operational  equipment?  And  (d)  Should  performance  on  the  device  be 
positively  related  to  performance  on  the  operational  equipment?  The 
responses  to  these  questions  and  the  detailed  comments  are  found  in 
Appendix  G. 

Figures  5-2  to  5-3  summarize  the  answers  to  questions  a  through  c. 
Answers  to  question  d  are  discussed  in  the  next  section.  Figure  5-2 
compares  all  four  devices  on  gunner  task  elements.  Figure  5-3  presents  a 
comparison  of  U-COFT  and  SIMNET  on  tank  commander  elements,  because  only 
these  two  devices  provide  for  tank  commander  participation  in  gunnery. 
Finally,  Figure  5-4  presents  a  summary  of  loader  and  driver  elements  for 
the  SIMNET  device,  the  only  device  that  provides  for  these  duty  stations. 
In  each  figure,  device  fidelity  is  described  by  two  related  ratings.  The 
first  rating  (round  symbols)  corresponds  to  the  concept  of  comprehensive¬ 
ness,  i.e.,  the  proportion  of  activity  elements  that  can  be  performed  or 
practiced  on  the  devices.  This  proportion  was  determined  by  dividing  the 
number  of  elements  which  can  be  performed  on  the  device  (question  a  above) 
by  the  total  number  of  elements.  The  second  rating  (square  symbols)  may 
be  interpreted  as  device  realism  in  that  it  describes  the  proportion  of 
performed/practiced  elements  that  are  not  subject  to  degraded  fidelity 
conditions.  This  second  rating  was  defined  then  as  the  number  of  elements 
without  missing  subcomponents  (question  b)  or  whose  stimulus  or  response 
components  were  not  altered  (question  c)  divided  by  the  number  of  elements 
that  may  be  practiced  (question  a).  Thus,  the  realism  rating  should  be 
interpreted  only  in  conjunction  with  comprehensiveness,  because  the  former 
rating  is  conditional  upon  the  latter.  For  instance  referring  to 
Figure  5-2,  VIGS  and  TopGun  receive  the  highest  realism  rating  for 
elements  of  the  PREFIRE  checks  that  it  simulates.  However,  the  comprehen¬ 
siveness  ratings  indicate  that  those  devices  only  a  few  of  the  elements 
can  be  simulated  at  all.  Thus,  device  fidelity  is  actually  quite  limited 
for  this  activity. 

Results  from  the  preceding  analyses  lead  to  a  number  of  generaliza¬ 
tions  concerning  device  capabilities  and  limitations.  We  begin  with  some 
generally  positive  statements  of  device  capabilities: 

1.  Over  the  entire  domain  of  gunnery  conditions  and  actions,  U-COFT 
provides  a  very  comprehensive  and  realistic  simulation  of  gunnery  condi¬ 
tions  and  actions.  Considering  the  purpose  and  the  cost  of  U-COFT,  this 
is  not  an  unexpected  result. 


& 

5$ 


kD 

LlJ 

5o 


□  an 


OOO  0099 


^  o>  is 

in  ~  c 

<o  5 
—  o)  ; , 


5  |  2 

J  1  CD 


c 
o 

CL)  — 

•S  £ 

O  </> 

H  09  O 

=»  ^  D- 

V  r  <-> 


O  L. 

v-  a) 

*“  CL 
‘  09 
Q.  09 

09  " 

jd  =n 
_  <D 
ZJ)  c 

o  C 

£  ~ 

r.  w 

u  O 
O  1/1 

o  ^1 


r  §  ^ 

01  w  o 

£  p  t 

43  o  g. 

1  Si 
§  i  t, 

£  I  e 

CD  Jrr  3 

Q.  £  O' 

CL  09 

©  09  U 

"  -Q  Of) 

=J>  -jj  C 

O  o  £ 

£  E  £ 


Q)  <o 

r_  w* 

«  ® 

£  e* 

a.  £ 


&? 

I 


CD 
C0  I  (ft 


CO  Z  Q 


0)  w 

o)  <2 


iOOOMElQ 


^*W5v!?vmVW 


A->WAvvvVN 


( 


g 

tjjfl 


Prepare  Stations  ror  Operation  (PREOPS) 


Perform  Prepare-to-Fire  (PREFIRE)  Checks 


Acquire  Target(s) 


Engage  Single  Target  from  the  Defense  Using  Precison  Gunnery 


Adjust  Fire 


Enga^d  a  Single  Target  with  the  Coax 


Enqaqe  Multiple  T eroets  with  the  Main  Gun 


Engage  Targets  with  the  Cal  .50 


Engage  Targets  Using  Battlesight  Gunnery 


Engage  Targets  Given  Fire  Control  System  Failure0 


Engage  Targets  Using  the  GAS 


Engage  Targets  in  Emergency  Mode 


Engage  Targets  In  Manual  Mode 


Engage  Target  from  the  TC  Position 


Assess  Results  of  Engagement 


0  Includes  the  following  failures:  ineffective  LRF,  multiple  returns  from  LRF,  loss  of 
GPS  symbology,  crosswind  sensor  failure,  cant  sensor  failure,  lead  angle  sensor  failure, 
GPS  failure,  and  GPS/TIS  failure. 

£  All  elements  (100%)  may  be  performed  on  the  device. 

Most  elements  (£50%)  may  be  performed  on  the  device. 

Some  elements  (<  50%)  may  be  performed  on  the  device. 

No  Symbol  No  elements  may  be  performed  on  the  device. 

H  None  of  the  elements  (0%)  that  may  be  performed  are  subject  to  reduced  fidelity. 

|  |  Some  of  the  elements  (£50%)  that  may  be  performed  are  subject  to  reduced  fidelity. 

Most  of  the  elements  (>  50%)  that  may  be  performed  are  subject  to  reduced  fidelity 
N/A  Device  does  not  simulate  key  conditions  required  to  perform  activity. 

Figure  5-3.  Summary  of  tank  commander  activities  supported  by  gunnery 
training  devices. 
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failure,  GPS  failure,  and  GPS/TIS  failure. 


O 


No  Symbol 


All  elements  (100  X)  may  be  performed  on  the  device. 
Most  elements  (>  50X)  may  be  performed  on  the  device. 
Some  elements  (<  50%)  may  be  performed  on  the  device. 
No  elements  may  be  performed  on  the  device. 


H 

□ 

N/A 


None  of  the  elements  (OX)  that  may  be  performed  are  subject  to  reduced 
fidelity. 

Some  of  the  elements  (^50%)  that  may  be  performed  are  subject  to  reduced 
fidelity. 

Most  of  the  elements  (>  50X)  that  may  be  performed  ore  subject  to  reduced 
fidelity 

Device  does  not  simulate  key  conditions  required  to  perform  activity 


Figure  5-4  Summary  of  loader  and  driver  activities  supported  by  SIMNET  device. 
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2.  Despite  its  avowed  purpose  to  train  tactical  skills,  SIMNET  can 
support  training  of  much  of  the  gunnery  domain.  In  terms  of  gunnery 
conditions,  SIMNET  provides  a  simulation  of  the  mission  requirement 
gunnery  parameters  (e.g.,  fire  control,  movement  formation)  that  are  not 
supported  at  all  by  the  other  simulators.  In  terms  of  actions,  SIMNET 
provides  a  sufficient  simulation  of  many  crew  gunnery  activities,  and  it 
is  the  only  computer-based  simulator  to  provide  practice  on  driver  and 
loader  activities. 

3.  The  low-cost  gunnery  devices  (VIGS  and  TopGun)  provide  adequate 
simulations  of  the  conditions  and  actions  related  to  precision  gunnery 
from  a  stationary  tank.  While  this  activity  is  but  a  small  part  of  the 
gunnery  domain,  precision  gunnery  skills  are  regarded  as  prerequisite  to 
many  of  the  other  aspects  of  gunnery. 

The  tables  point  to  some  general  device  limitations  as  well.  It 
should  be  noted  that  many  of  these  negative  comments  represent  "old  news" 
simply  restating  what  may  be  obvious  to  those  who  work  with  these  devices. 
Other  comments  are  not  necessarily  indictments  of  devices  because  they  may 
not  have  been  designed  to  train  objectives  within  the  presently  defined 
gunnery  domain.  In  other  words,  the  following  device  limitations  may  be 
interpreted  as  deficiencies  only  with  respect  to  our  proposed  training 
program. 

1.  VIGS,  TopGun,  and  SIMNET  allow  crewmembers  to  perform  few  if  any 
activities  related  to  secondary  gunnery  functions  (PREOPS  and  PREFIRE 
procedures).  Furthermore,  although  U-COFT  does  provide  some  simulation  of 
these  activities,  it  also  has  potential  sources  for  negative  transfer. 
These  sources  of  negative  transfer  are  discussed  in  the  next  section. 

2.  In  general,  fidelity  measures  for  all  devices  are  low  with 
respect  to  target  acquisition  activities.  These  poor  ratings  are  partly 
due  to  the  fact  that  no  simulator  simulates  open  hatch  viewing.  Tank 
Combat  Tables  (FM  17-12-1)  states  that  "during  buttoned-up  [closed  hatch] 
operations,  the  tank  crew's  ability  to  acquire  targets  is  reduced  by  at 
least  50  percent..."  (p.  3-3).  Therefore,  the  preferred  mode  for  target 
acquisition  is  not  simulated  by  any  of  the  devices. 

3.  Whereas  many  of  elements  of  driving  and  loading  are  simulated  by 
SIMNET,  a  substantial  portion  must  be  performed  under  reduced  fidelity. 

In  that  regard,  Bessemer  (1986)  evaluated  SIMNET  with  respect  to  driver 
activities.  He  concluded  that  whereas  fidelity  of  SIMNET  driving  could  be 
sufficient  to  support  tactical  movement  and  manuever,  there  were 
significant  fidelity  problems  to  invalidate  its  use  for  training  and 
sustaining  basic  fine  driver  control  skills.  Similar  fidelity  limitations 
affect  SIMNET's  simulation  of  loading.  For  instance,  the  act  of 
physically  removing  a  round  from  the  ready  rack  and  loading  it  into  the 
main  gun  tube  is  simulated  by  a  series  of  button  pushes.  We  conclude, 
therefore,  that  SIMNET  is  not  appropriate  for  initial  training  of  either 
loader  or  driver  skills. 
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4.  As  indicated  by  Figure  5-1  and  the  not  applicable  (N/A)  symbols 
in  Figures  5-2  to  5-4,  some  important  parameters  of  the  gunnery  domain  are 
not  simulated  by  the  devices: 

.  VI6S  and  TopGun  do  not  simulate  owntank  movement.  Therefore,  they 
cannot  provide  training  in  engaging  targets  from  a  moving  tank,  an 
important  offensive  gunnery  technique. 

.  VIGS,  TopGun,  and  SIMNET  provide  little  if  any  practice  on  degraded 
mode  gunnery  techniques.  U-COFT  simulates  some  of  the  important 
degraded  gunnery  techniques  such  as  battlesight  gunnery,  engaging 
targets  using  the  gunner's  auxiliary  sight  (GAS),  and  engaging 
targets  in  emergency  and  manual  modes.  However,  even  U-COFT  fails  to 
provide  simulation  of  the  following  fire  control  system  failures: 
loss  of  GPS  symbology,  crosswind  sensor  failure,  cant  sensor  failure, 
and  lead  angle  sensor  failure. 

.  Not  all  weapon  systems  are  simulated.  The  M240  coaxial  machine  gun 
is  not  represented  by  TopGun  and  SIMNET,  and  Cal  .50  machine  gun  is 
not  represented  on  VIGS,  TopGun,  and  SIMNET.  These  deficits  severely 
restrict  the  types  of  engagements  that  can  be  trained  on  the  devices. 

Specific  Sources  of  Negative  Transfer  and  Correlations.  As  explained 
above,  the  analysis  of  gunnery  actions  identified  activity  elements  in 
which  positive  transfer  or  correlation  between  device  and  tank  was  not 
expected  (evaluation  question  d  in  the  previous  section).  In  some  cases, 
positive  relations  were  not  predicted  because  the  device  is  so  unlike  the 
actual  equipment.  The  focus  of  this  section,  however,  is  on  a  subset  of 
those  instances  of  nontransfer  for  which  there  are  reasons  to  predict 
negative  transfer  and/or  negative  correlations.  We  realize  that  the 
effects  of  any  negative  transfer  or  correlation  source  may  be  mitigated  by 
appropriate  instruction  or  control  of  practice  events.  Thus,  the 
following  problems  must  be  regarded  as  potential  sources  of  negative 
transfer  or  correlation. 

We  begin  this  discussion  with  negative  sources  of  transfer  or 
correlation  that  are  general  in  that  they  apply  to  more  than  one  device: 

1.  U-COFT  and  VIGS  exercises  start  with  some  sort  of  preview  that 
includes  a  description  of  the  to-be-engaged  target(s)  and  the  conditions 
under  which  the  engagement  is  to  be  conducted.'  These  previews  encourage 
the  student  to  preset  switches  on  the  Fire  Control  Panel  according  to  the 
exercise  conditions.  Students  could  learn  to  depend  on  this  sort  of 
preview  information,  which  is  certainly  not  available  in  a  tactical 
gunnery  situation.  Most  importantly,  the  presence  of  a  preview  encourages 
the  student  to  arm  the  main  gun/coax  and  the  laser  range  finder  before 
receiving  a  fire  command  from  the  TC.  The  unsafe  habit  of  presetting 

'’Target  previews  on  the  Ml  VIGS  are  only  presented  on  the 

Perceptronics  demonstration  videodisc,  not  on  the  ARI  experimental  thermal 

videodisc. 
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these  switches  in  the  tank  can  lead  to  accidental  firings  of  the  laser  or 
weapon  resulting  in  injury  or  death. 


2.  VIGS,  TopGun,  and  U-COFT  do  not  simulate  the  loader's  station, 
and  none  of  the  devices  (including  SIMNET )  simulate  main  gun  recoil. 

Thus,  they  do  not  provide  the  negative  consequences  associated  with  firing 
the  gun  too  quickly.  Boldovici  (in  press)  speculated  that  this  specific 
fidelity  deficit  plus  scoring  systems  that  place  a  premium  on  quick 
responding  result  in  gunnery  training  that  overemphasizes  speed  at  the 
expense  of  accuracy  or  safety  considerations.  Improper  loading  or  firing 
before  the  loader's  "UP"  announcement  can  result  in  serious  injury  to  the 
loader. 

3.  The  three  devices  that  employ  computer-generated  imagery  (TopGun, 
U-COFT,  and  SIMNET)  do  not  provide  appropriate  texture  gradient  and  aerial 
perspective  cues  for  perception  of  depth.®  The  result  is  that  targets 
appear  closer  causing  ranges  to  be  underestimated.  These  misleading  cues 
are  a  potential  negative  correlation  source  because  experts  would  be  more 
likely  to  use  these  depth  cues  than  novices.  Anecdotal  reports  indicate 
that  experienced  crewmen  can  quickly  adjust  to  the  range  distortions, 
apparently  ignoring  these  cues  and  attending  to  other  monocular  depth 
perception  cues  that  are  not  distorted  by  computer-generated  displays: 
apparent  size,  interposition,  and  movement  perspective. 9 

4.  All  devices  present  a  limited  array  of  targets  that  are  easily 
distinguishable.  The  actual  battlefield  will  be  filled  with  a  variety  of 
friendly  and  threat  targets  that  are  easily  confused,  e.g.,  Soviet  T72 
and  French  AMX  tanks.  Further,  devices  present  full  as  opposed  to  partial 
target  profiles,  so  that  students  receive  no  practice  at  identifying 
targets  with  incomplete  information.  Consequently,  crewmen  may  fail  to 
develop  critical  skills  needed  to  identify  targets  under  battlefield 
conditions. 

5.  VIGS  and  SIMNET  do  not  simulate  the  operation  of  the  lead  sensor 
system.  This  limitation  may  result  in  the  gunner  acquiring  inappropriate 
behaviors  related  to  tracking  moving  targets  and  tracking  stationary 
targets  from  a  moving  tank.  For  one,  the  student  may  fail  to  learn  to 
dump  the  lead  solution  when  the  target  reverses  direction  or  when  moving 


®Texture  gradient  refers  to  the  changes  in  the  perceived  density  of 
detail  that  varies  inversely  with  distance.  Aerial  perspective,  on  the 
other  hand,  refers  to  systematic  changes  in  color  as  a  function  of 
distance,  i.e.,  distant  objects  are  less  saturated  than  close  ones  and 
tend  to  take  on  the  color  of  ambient  light. 

^Apparent  size  refers  to  the  perception  of  distant  objects  as  smaller 
than  closer  ones  that  are  equal  in  physical  size.  Interposition  is  the 
perceptual  phenomenon  where  objects  in  the  foreground  block  objects  in  the 
background  from  view.  Movement  perspective  is  experienced  when  distant 
objects  appear  to  displace  more  slowly  than  close  objects  when  the  objects 
move  or  when  the  observer  changes  perspective. 


to  a  different  target.  Also,  he  may  not  learn  to  track  targets  for 
required  period  of  time  prior  to  firing.  Finally,  he  may  learn  to  ambush 
targets  ( i . e . ,  fire  whenever  the  moving  target  crosses  the  reticle  aiming 
point)  without  having  input  the  correct  lead.  In  this  regard,  Kraemer  and 
Bessemer  (1987)  provided  evidence  of  negative  transfer  from  SIMNET 
training  to  performance  in  the  Canadian  Army  Trophy  (CAT)  competition 
resulting  from  this  source. 

The  following  negative  sources  of  transfer  and  correlation  apply  to 
certain  individual  devices: 

1.  SIMNET  color  codes  targets  to  distinguish  friendly  from  threat 
vehicles  (friend  =  tan,  threat  =  green).  This  extra  information  in  the 
display  makes  the  process  of  identifying  friend,  foe,  or  neutral  targets 
(IFFN)  trivially  easy  to  perform.  However,  the  student  may  not  learn  to 
make  the  IFFN  determination  on  the  basis  of  target  identification  as 
required  in  the  battlefield.  Such  an  effect  is  analogous  to  the  "crutch" 
effect  cited  in  the  experimental  learning  literature  (Bilodeau,  1952, 
cited  in  Boldovici,  1979).  To  demonstrate  a  crutch  effect,  an 
experimental  group  is  presented  some  extra  informational  feedback  that 
aids  task  performance.  When  transferred  to  the  same  task  but  without  such 
extra  feedback,  their  performance  is  inferior  with  respect  to  a  control 
group  who  learned  to  perform  the  task  in  absence  of  the  extra  information. 
Although  originally  defined  with  respect  to  informational  feedback, 
Boldovici  (1979)  extended  the  concept  of  a  performance  "crutch"  to  other 
salient  stimuli  that  control  task  performance. 

2.  TopGun  color  codes  targets  with  respect  to  engagement  priority 
(red  =  most  dangerous,  yellow  =  dangerous,  blue  =  least  dangerous)  in  the 
Wide  Field  of  View  area  of  the  display.  Because  these  extra  cues  are  not 
available  in  tactical  gunnery,  the  potential  exists  for  a  crutch  effect  as 
described  above.  Specifically,  the  gunner  may  not  learn  to  classify 
targets  as  he  would  on  the  battlefield,  i.e,  on  the  basis  of  target 
behavior,  lethality,  and  range. 

3.  As  performed  on  U-COFT,  the  PREFIRE  activities--particularly 
those  related  to  boresighting--are  substantially  different  from  those 
outlined  in  FM  17-12-1.  Consequently,  the  student  may  learn  incorrect 
and/or  suboptimal  boresighting  procedures.  For  instance,  U-COFT 
boresighting  requires  the  gunner  to  take  only  one  set  of  boresight 
readings  that  are  directly  input  into  the  ballistic  computer.  Ml 
procedures  (FM  17-12-1)  call  for  the  gunner  to  take  two  readings,  compute 
an  average,  and  input  the  average  into  the  ballistic  computer.  The 
problems  associated  with  boresighting  in  the  U-COFT  are  detailed  by 
Morrison  (1987). 

4.  The  thermal  image  in  the  U-COFT  is  a  simple  translation  of  the 
day  channel  image.  Therefore,  it  does  not  provide  any  extra  information 
than  the  actual  Ml  sight  provides.  This  extra  information  includes 
signatures  such  as  hot  gun  tubes  or  tracks  that  can  be  used  to  acquire 
targets  but  would  otherwise  be  unobservable  through  the  day  channel. 
Research  indicates  that  alternating  viewing  between  thermal  and  optical 
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sights  increases  the  probability  of  target  detection  (Kottas  &  Bessemer, 
1983)  Indeed,  anecdotal  reports  from  U-COFT  instructors  indicate  that 
experienced  gunners  often  search  day  scenes  with  both  channels  in  the  U- 
COFT.  However,  after  it  becomes  clear  to  the  student  that  there  is  no 
additional  information  in  the  U-COFT  thermal  sight,  this  appropriate 
alternating  behavior  occurs  less  frequently. 

5.  To  score  a  hit  on  the  prototype  Ml  V I GS ,  the  reticle  must  be  on 
the  target  at  the  time  of  impact.  The  result  is  that,  unlike  the  actual 
equipment,  control  handle  inputs  can  affect  the  fall  of  the  round  after 
the  shot  has  been  fired.  The  student  gunner  may  acquire  both  good  and  bad 
habits  as  a  result  of  this  device  logic.  On  the  positive  side,  the 
student  learns  the  appropriate  behavior  of  continuing  to  track  after 
firing,  a  strategy  that  increases  the  probability  of  observing  round 
effects  and  achieving  a  second  round  hit.  Unfortunately,  he  may  also 
learn  to  fire  quickly  without  a  correct  sight  picture,  and  then  to  adjust 
the  reticle/target  relationship  as  the  round  is  in  flight.  The  latter 
behavior  is  clearly  inappropriate  for  tank  gunnery. 


Conclusions 


In  this  section,  we  summarize  the  results  of  the  device  evaluation  by 
presenting  suggested  short-  and  long-range  solutions  to  some  of  the  more 
serious  fidelity  problems.  Short-range  solutions  are  changes  to  the  way 
we  use  training  media  and  devices.  These  changes  will  be  incorporated  in 
the  proposed  training  program.  Long-range  solutions  involve  extensive 
software  changes  or  training  device  development,  both  of  which  are  beyond 
the  scope  of  the  present  project. 

Safety-related  Problems.  Two  problems  were  described  which  indicated 
that  gunnery  devices  overemphasize  speed  of  responding  to  the  detriment  of 
safety  concerns.  The  short-range  solution  is  to  train  instructors  to  be 
vigilant  in  detecting  safety-related  errors  such  as  presetting  fire 
control  switches  or  firing  before  the  loader's  UP  announcement. 
Unfortunately,  the  low-cost  trainers  (VIGS  and  TopGun)  are  designed  to  be 
used  with  little  or  no  instructor  intervention.  The  longer  term,  and 
perhaps  more  satisfactory,  solution  is  to  upgrade  device  capabilities  to 
detect  such  errors  and  to  penalize  students  accordingly. 

Secondary  Functions.  Gunnery  devices  are  not  now  capable  of  training 
the  secondary  functions  of  PREOPS  and  PREFIRE  procedures.  One  solution  is 
to  develop  a  procedures  trainer  for  those  purposes.  However,  secondary 
function  training  can  be  accomplished  on  the  actual  tank  without  incurring 
costs  related  to  going  to  the  range  and  firing  live  ammunition.  For 
instance,  training  on  these  tasks  can  be  accomplished  in  areas  such  as  the 
motor  pooi.  In  summary,  the  tank  itself  is  still  probably  the  best  medium 
for  training  these  tasks. 

Loader  and  Driver  Skills.  At  the  present  time,  there  are  no  devices 
that  address  the  initial  acquisition  of  driver  and  loader  skills  related 
to  tank  gunnery.  However,  tank  driver  trainers  (TDTs)  are  proposed  to  be 
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implemented  in  institutional  training  for  both  the  M60-  and  Ml-series 
tanks  in  the  near  future  (U.S.  Army  Armor  School,  1987).  Designed  for 
initial  training  on  driving  skills,  the  TDTs  will  consist  of  a  realistic 
driving  compartment  that  presents  real  time  visual  and  motion  cues 
associated  with  standard,  degraded,  and  emergency  conditions.  There  are 
no  corresponding  plans  to  produce  a  loader  trainer.  One  could  argue  that 
loading  tasks  involve  primarily  gross  motor  skills  that  would  not  be  cost 
effectively  trained  by  a  computer-based  device.  On  the  other  hand, 
loading  a  round  on  a  moving  tank  may  require  a  certain  degree  of  fine 
motor  skill.  Research  should  be  performed  to  more  precisely  define  the 
performance  requirements  of  loading  in  order  to  determine  the  need  for  a 
loader  trainer. 

Target  Acquisition.  Target  acquisition  training  is  not  well 
supported  by  the  present  devices.  The  short-term  solution  is  to  design  a 
field-based  target  acquisition  course  that  reduces  the  use  of  operational 
equipment  and  increases  the  student's  opportunities  to  practice  the 
decisions  required  in  target  acquisition.  A  long-term  solution  would  be 
to  design  and  build  a  target  acquisition  part-task  trainer  that  allows 
practice  on  these  activities  without  going  to  the  field.  The  training 
objectives  for  the  part-task  trainer  could  address  some  or  all  of  the 
specific  deficiencies  noted  in  this  report:  (a)  range  estimation,  (b) 
target  identification,  (c)  target  classification,  (d)  thermal  vs.  optical 
channels  viewing,  and  (d)  open-  vs.  closed-hatch  viewing.  The  trainer 
would  require  high  fidelity  with  respect  to  the  visual  display.  However, 
because  the  objective  is  to  practice  decision-making  skills  related  to 
target  acquisition,  response/control  fidelity  could  be  only  minimal.  For 
instance,  the  output  decisions,  such  as  target  engagement,  could  be 
indicated  by  a  button  push  or  the  like.  A  possible  medium  for  this 
trainer  could  be  an  intelligent  ( i . e . ,  computer-controlled)  videodisc 
system  with  high  resolution  video. 

Degraded  Mode  Gunnery.  Although  the  U-COFT  addresses  nonprecision 
gunnery,  it  does  not  provide  practice  on  every  aspect  of  degraded  mode 
gunnery.  This  may  not  be  critical,  however,  as  our  domain  indicates  that, 
other  than  battlesight  and  GAS  gunnery,  degraded  mode  gunnery  consists  of 
relatively  few  unique  actions  that  must  be  practiced.  The  difficulty  in 
employing  degraded  techniques  is,  therefore,  not  in  the  performance  of 
those  techniques  but  in  the  recognition  of  the  conditions  under  which  they 
are  used.  In  other  words,  performance  problems  related  to  degraded  mode 
gunnery  reflect  knowledge  rather  than  skill  deficiencies.  In  that  regard, 
ARI  and  the  Training  Technology  Field  Activity  (TTFA)  have  developed 
courseware  on  degraded  mode  gunnery  for  implementation  on  the  Hand-Held 
Tutor  (HHT).  The  HHT  is  a  low-cost,  electronic  device  for  providing  drill 
and  practice  on  selected  Army  technical  training  topics.  Hoffman  (1987) 
has  noted  some  inaccuracies  in  the  HHT  courseware.  Nevertheless,  the  HHT 
is  a  useful  adjunct  device  to  promote  acquisition  of  knowledges  related  to 
degraded  mode  gunnery. 
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V 


Instructional  Features 


Instructional  features  are  generally  defined  as  hardware  and  software 
capabilities  that  support  the  process  of  instruction  in  computer-based 
training  devices.  At  least  seven  research  reports  have  attempted  to 
inventory  and  define  instructional  features  (Caro,  Pohlman,  &  Isley,  1979; 
General  Electric,  1983;  Logicon,  Inc.,  1985;  Hughes,  1979;  Pozella,  1983; 
Semple,  Cotton,  &  Sullivan,  1981;  and  Sticha  et  al . ,  1986).  Appendix  H 
summarizes  this  literature  by  describing  each  feature  in  terms  of  its 
function,  its  training  purpose,  and  other  references  in  the  research 
literature.  Two  generalizations  may  be  drawn  from  this  literature. 

First,  many  of  the  features  are  specifically  oriented  to  flight  training 
applications,  e.g.,  crash  override.  This  observation  reflects  the  fact 
that  the  initial  research  on  instructional  features  was  performed  in  the 
context  of  flight  training.  Nevertheless,  there  is  one  review  (General 
Electric,  1983)  that  is  in  fact  oriented  to  development  of  an  armor 
training  device--the  Institutional  Conduct  of  Fire  Trainer  (I-COFT). 
Second,  there  are  differences  among  these  research  reports  in  the  contents 
of  their  inventories.  Appendix  H  indicates  the  relative  agreement  by 
listing  features  in  order  of  the  number  of  associated  references.  The 
features  with  more  references  are  more  general  in  function  (i.e.,  less 
dependent  on  the  nature  of  training),  whereas  features  with  fewer 
references  were  more  idiosyncratic  in  nature  (i.e.,  relevant  to  a 
particular  training  application).  We  expected  that  armor  gunnery  devices 
would  employ  some  of  the  conventional,  general  purpose  instructional 
features  as  well  as  those  oriented  specifically  to  armor  gunnery 
applications. 

Some  of  the  instructional  features  listed  in  Appendix  H  are 
specifically  designed  to  measure  performance  rather  than  to  facilitate 
training  per  se.  Such  features  include  automated  performance  measurement, 
hardcopy/printout,  and  data  storage/analysis.  These  capabilities  are 
viewed  as  instructional  features  because  they  off-load  performance 
measurement  as  an  instructor  duty.  However,  the  focus  of  the  present 
section  is  on  instructional  features  that  facilitate  individual  skill 
acquisition  or  the  process  of  training  on  the  device.  Performance 
measurement  capabilities  are  examined  in  detail  in  Chapter  6. 

Much  of  the  research  on  the  effectiveness  of  instructional  features 
has  been  performed  by  Hughes  and  his  associates  in  the  context  of  flight 
simulation.  In  reviewing  this  literature,  Sticha  et  al.  (1986)  noted 
results  from  three  studies.  Hughes,  Hannaman,  and  Jones  (1979)  showed 
that  use  of  automated  demonstration  and  record/replay  features  was  shown 
to  be  less  beneficial  than  a  single  practice  trial.  Hughes,  Lintern, 
Wightman,  and  Brooks  (1981)  demonstrated  that  use  of  freeze  and  reset 
features  did  not  significantly  increase  performance.  In  contrast  to  the 
two  previously  cited  studies,  Bailey,  Hughes,  and  Jones  (1980)  found 
significant  performance  improvement  as  a  result  of  using  the  initial 
conditions  feature  to  provide  backward  chaining  schedule  for  training  a 
30-degree  dive  bomb  task.  However,  Sticha  et  al.  cautioned  the  reader  to 
avoid  implicitly  accepting  the  null  hypothesis: 
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It  would  be  inappropriate  to  conclude  from  these  experiments 
that  the  initial  conditions  instructional  feature  is  effective 
and  that  the  automated  simulator  demonstration,  record/replay, 
freeze,  and  reposition  instructional  features  are  not.  It  is 
quite  likely  that  the  training  efficiency  of  an  instructional 
feature  is  largely  dependent  upon  the  manner  in  which  it  is 
employed.  Thus,  a  more  appropriate  interpretation  of  these  data 
emphasizes  the  positive  results  of  Bailey  et  al.  (1980):  Their 
experiment  showed  that  the  initial  conditions  instructional 
feature  can  provide  significant  training  benefits  if  combined 
with  an  effective  training  technique  such  as  backward  chaining. 

(p.  64) 

Although  the  literature  on  instructional  features  is  not  extensive, 
this  review  indicates  that  features  can  be  effective  given  an  appropriate 
application.  Accordingly,  our  analysis  of  instructional  features  not  only 
identified  the  features  that  are  incorporated  on  current  devices  but  also 
specified  how  they  may  be  used  to  determine  their  effectiveness. 


Evaluation  of  Instructional  Features 

As  a  first  step,  we  identified  the  instructional  features  available 
on  each  device.  The  list  of  instructional  features  in  Appendix  H  was  used 
as  a  guide  in  determining  device  capabilities.  The  technical  literature 
on  the  devices  was  also  helpful  in  determining  device  capabilities. 
However,  the  device  literature  often  used  unique  terms  to  describe  device 
features.  To  facilitate  comparisons  across  devices,  we  used  more  common 
feature  names  as  given  in  Appendix  H  to  describe  the  device  functions.  As 
expectpd,  however,  examination  of  the  devices  revealed  special  purpose 
instructional  features  not  in  Appendix  H.  These  more  idiosyncratic 
features  were  given  names  that  were  descriptive  of  their  functions. 

Table  5-1  provides  a  summary  of  the  armor  device  instructional  features 
including  a  short  description  of  how  each  feature  is  implemented  on 
particular  devices.  In  all,  17  separate  features  were  identified  over  all 
4  devices.  Of  the  17,  12  instructional  features  were  found  implemented  on 
U-COFT  in  comparison  with  9  on  TopGun,  7  on  VIGS,  and  4  on  SIMNET. 

Table  5-1  also  organizes  features  into  trainer  functions  that  they 
are  intended  to  support.  Similar  taxonomic  schemes  have  been  used  in  the 
literature  (e.g.,  Semple  et  al.,  1981;  Logicon,  Inc.,  1985).  This 
organization  was  also  used  to  explicate  the  functions  of  features  and  to 
structure  the  following  comments.  Included  in  the  comments  are  our 
speculations  as  to  how  the  feature  is  or  should  be  used  to  affect 
learning. 
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Table  5-1 


Training  Features  as  Implemented  on  the  Gunnery  Training  Devices 


Training  Function 
Training  Feature 


_ Gunnery  Training  Devices _ 

VIGS  TopGun  U-COFT  SIMNET 


Plan  Training 


Scenario  Control  Selects  exercises  Creates  and  stores 

according  to  a  set  of  exercises  according 
3  parameters.  to  a  set  of  8 

parameters. 


Selects  exercises  Creates  scenarios 

according  to  a  set  of  according  to  11 
9  parameters.  parameters. 


Exercise  Sequence 
Control 


Manual ly  sequences 
selected  exercises. 


Automatically 
generates  exercise 
parameters  according 
to  two  factors: 

.  student  proficiency 
level,  and 
.  stage  of 
learning. 


Automatically 
sequences  students 
through  a  "training 
matrix"  derived  from 
three  dimensions  of 
difficulty: 

.reticle  aim, 

.target  acquisition, 
and 

.system  management. 


N/A 


Control  Practice 


Record/Replay  N/A 


Freeze/Unfreeze  N/A 


Records  and  replays 
visual  cues  from  an 
exercise.9 


Records  and  replays 
visual  and  aural  cues 
(except  for  crew 
communications)  from 
an  exercise. 


Records  missions 
for  replay  on 
Plan  View  Display 
(see  Remote 
Graphics  Replay). 


Freezes/unfreezes 
action  in  present 
exercise.3 


Freezes/unfreezes 
action  in  present 
exercise. 


N/A 


Reset 


Kill  Override 


N/A 


N/A 


Automatically  resets 
to  beginning  of 
current  stage  if 
player  is  killed  but 
has  remaining 
"tanks. (See  Kill 
Override) 


Restarts  (repeats) 
any  exercise  that  has 
been  selected. 


N/A 


Allows  player  three  N/A 

"tanks"  (lives);  if 

any  remain,  player  is 

reset  after  being 

killed. 


Recons ititues  tank 
at  point  of  kill 
or  any  other 
location  in  the 
data  base  specified 
by  the  controller. 


(table  continues) 


_ Gunnery  Training  Devices _ 

Training  Feature  VIGS  TopGun  U-CQFT  SIMNET 


Administer/Monitor 

Training 


demote 

Instructor/ 

Operator  Station 
(IOS) 

N/A 

N/A 

Controls  operation  of 
U-COFT,  provides 
variety  of  on-line 
performance  data,  and 
provides  continuous 
viewing  of  sight 
pictures. 

N/A 

Procedures 

Monitoring 

N/A 

N/A 

System  Set-Up 
specifies  which 
switches  should  be 
monitored  and  any 
switch  position 
errors  that  occur  at 
the  beginning  of  the 
exercise. 

N/A 

Computer-Managed 

Instruction 

N/A 

N/A 

Keeps  track  of 
individual,  crew,  and 
unit  progress  through 
the  training  matrix. 

N/A 

Briefing  Utilities 

Adjust  Gunnery 

Parameters 

N/A 

N/A 

Provides  a  written 
briefing  on  the 
upcoming  exercise  for 
the  instructor  to 

deliver  to  the 
student. 

Ammunition  Load 

Varies  the  maximum 

number  of  rounds  that 
may  be  fired  in  a 
single  engagement. 

Varies  initial  tank 

aimunition  load  that 

is  available  for  the 
entire  game. 

N/A 

Varies  the  number  of 
main  gun  anno  rounds 
available  for  the 
simulated  mission.0 

Variable  Loading 
Interval 

Varies  the  interval 

from  the  fire  command 

to  UP  announcement. 

Varies  the  interval 

from  the  fire  command 
to  the  UP  announce- 

N/A 

N/A 

ment 


(table  continues) 


Training  Feature 


VIGS 


Gunnery  Training  Devices 
TopGun 


U-COFT 


SIMNET 


Kill  Zone 

Varies  the  criteria 
for  hitting  a  target 
from  5%  to  300%  of 
the  target  profile. 

Varies  the  criteria 
for  hitting  a  target 
up  to  100%  of  target 
profile. 

N/A 

N/A 

Reticle 

Misal ignment 

Misaligns  aiming 
point  with  fal 1  of 
round  causing  first- 
round  miss. 

N/A 

N/A 

N/A 

Simulate  Events 

Automated 

Crewmembers 

Simulated  TC  provides 
fire  commands  and  LDR 

announces  UP. 

Simulated  TC  provides 
fire  command  and  LDR 

announces  UP.  TC  can 

also  override  GNR's 

controls  to  "slew" 

turret  toward  most 
dangerous  target. 

Simulated  LDR 
announces  up  and  DVR 
responds  to  MOVE  OUT 
and  STOP  commands. 

N/A 

Malfunction 

Selection 

N/A 

Can  simulate  "GPS 
out"  condition  by 
presenting  GAS 
reticle. 

Selects  exercises 
containing  desired 
malfunctions. d 

N/A 

Amro  Select/ 
Reload  Keys 

N/A 

N/A 

Initiates  simulation 
of  loading/reloading 
round  Including  aural 
cues  and  ballistic 

characteristics  of 
round  when  fired. 

N/A 

Note.  The  symbol  N/A  indicates  that  the  instructional  feature  is  not  available  on  the  device. 
aThis  feature  is  only  available  on  TopGun  models  equipped  with  the  TC  station. 


\ 

^TopGun  models  equipped  with  TC  stations  will  allow  the  instruction/TC  to  reset  the  game  to  the  beginning  of  the 
current  stage  or  to  the  beginning  of  the  next  stage. 


c0ther  rounds  may  be  received  during  mission  from  resupply  vehicle.  Ammunition  load  is  one  of  the  conditions 
required  to  specify  a  scenario. 

Malfunction  Selection  may  be  regarded  as  a  subfunction  of  Scenario  Control  rather  than  a  separate  training 
feature  because  malfunctions  may  be  accessed  only  through  selection  of  appropriate  exercises. 


Prepare  Training.  Two  of  the  features  provide  the  instructor  the 
ability  to  prepare  meaningful  exercises  on  the  training  device.  The  two 
features  address  the  creation  and  sequencing  of  training  exercises. 

1.  Scenario  control.  There  are  two  fundamentally  different  versions 
of  scenario  control  as  implemented  on  the  gunnery  training  devices.  In 
the  first  version,  as  implemented  on  VIGS  and  U-COFT,  the  instructor 
selects  features  from  a  finite  set  of  "canned"  exercises  according  to  a 
set  of  parameters.  In  the  second,  as  implemented  on  TopGun  and  SIMNET, 
the  instructor  can  "create"  virtually  an  unlimited  number  of  scenarios 
from  a  similar  but  longer  set  of  parameters.  Whereas  the  latter  version 
of  scenario  control  is  capable  of  presenting  a  wider  variety  of  exercises, 
we  suspect  the  former  version  is  easier  to  use.  However,  TopGun  has  the 
capability  of  storing  parameters,  making  it  easy  to  recreate  a  complicated 
exercise.  SIMNET  has  no  such  corresponding  feature. 

2.  Exercise  sequence  control.  The  second  feature  in  this  category 
allows  the  instructor  to  select  and  sequence  preprogrammed  exercises 
according  to  some  scheme  of  instruction.  VIGS  requires  the  instructor  to 
manually  sequence  selected  exercises  prior  to  a  training  session,  whereas 
TopGun  and  U-COFT  automatically  sequence  exercises.  Although  the 
automatic  feature  would  appear  more  labor-saving  than  the  manual  version, 
the  sequencing  method  implemented  on  TopGun  and  U-COFT  may  not  be  valid. 
The  sequencing  algorithms  are  not  based  on  any  discernable  instructional 
theory  or  practice;  rather  they  appear  to  be  based  on  some  implicit  notion 
of  task  difficulty.  Finally,  because  SIMNET  has  no  ability  to  store 
initial  exercise  conditions,  it  has  no  provision  for  automatically  or 
manually  sequencing  exercises. 

Control  Practice.  In  addition  to  setting  up  a  series  of  exercises, 
the  instructor  also  needs  on-line  control  of  practice  in  order  to 
facilitate  learning.  The  following  features  allow  the  instructor  to 
control  certain  practice  events  designed  to  speed  learning. 

1.  Record/replay.  To  learn  any  task,  a  student  must  have  some 
knowledge  of  his  task  performance.  One  potentially  effective  method  for 
providing  at  least  some  of  that  feedback  is  through  a  replay  of  stimulus 
events  that  occurred  during  performance  of  an  exercise.  Whereas  there  are 
reasons  to  doubt  the  effectiveness  of  passive  viewing,  replay  with  expert 
interpretation  is  potentially  a  powerful  pedagogical  technique.  TopGun 
and  U-COFT  both  provide  record/replay  features  whereas  VIGS  does  not. 
SIMNET  provides  a  form  of  replay  through  its  Plan  View  display.  This 
display  provides  a  top-down  view  of  all  or  a  subset  of  simulated  vehicles 
in  the  data  base  including  text  information  on  each  vehicle  (e.g., 
location,  speed,  orientation).  Unfortunately,  this  display  is  oriented 
more  towards  tactical  training  than  towards  gunnery  training. 

2.  Freeze/reset.  A  common  approach  to  training  difficult  tasks  is 
to  isolate  the  difficult  task  elements  and  provide  repetitive  practice  on 
those  particular  elements.  The  freeze/unfreeze  and  reset  features  make  it 
possible  to  provide  this  potentially  effective  form  of  practice.  The 
freeze  feature  can  also  be  used  to  interrupt  training  to  provide  coaching 


comments.  The  effectiveness  of  this  application  would  depend  on  the 
detrimental  effects  of  interrupting  training  vs.  the  quality  of  the 
coaching  comments.  Freeze/unfreeze  and  reset  features  are  implemented  on 
TopGun  and  U-COFT,  whereas  they  are  not  represented  on  VIGS  and  SIMNET. 

3.  Kill  override.  Frequent  simulated  kills  of  the  student's  tank 
can  interrupt  the  "flow"  of  training.  A  kill  override  feature  can  be  used 
to  eliminate  or,  at  least,  to  minimize  such  interruptions.  Kill  override 
should  be  used  in  the  early  stages  of  learning  where  the  probability  of  a 
kill  is  highest,  but  the  student  should  experience  more  realistic 
mortality  effects  as  he  gains  experience.  This  feature  is  notably  absent 
in  U-COFT,  wherein  exercises  end  whenever  the  owntank  is  killed.  On  the 
other  hand,  kill  override  is  somewhat  irrelevant  to  VIGS  where  owntanks 
cannot  be  killed. 

Administer  and  Monitor  Training.  The  following  instructional 
features,  all  implemented  on  U-COFT,  are  designed  to  aid  the  instructor  in 
administering  and  monitoring  training. 

1.  Remote  instructor/operator  station  and  procedures  monitoring. 

Two  features  aid  the  instructor  in  monitoring  the  process  of  instruction. 
The  remote  instructor/operator  station  allows  the  instructor  to  closely 
monitor  U-COFT  performance  while  not  being  in  the  student  compartment  with 
him.  There  is  less  need  for  this  feature  in  VIGS  and  TopGun  where  student 
performance  is  more  observable.  On  the  other  hand,  SIMNET  would  greatly 
benefit  from  these  features  because  student's  perform  within  compartments 
where  direct  observation  is  impossible.  The  procedures  monitoring 
function  (called  "system  set-up"  in  the  U-COFT  manuals)  allows  the 
instructor  to  monitor  switch  positions  via  the  I/O  station. 

2.  Computer-managed  instruction.  U-COFT  automatically  maintains 

records  of  crew  and  unit  performance  and  makes  it  accessible  to  training 
managers.  The  problem  with  this  feature  is  that  performance  information 
is  provided  with  reference  to  the  three-dimensional  matrix  model  of 
training  progression.  If  the  trainer  uses  some  other  instructional  model, 
the  records  are  not  particularly  useful.  x 

3.  Briefing  utilities.  The  U-COFT  presents  via  the  instructor/ 
operator  station  display  a  briefing  to  be  read  to  the  student  concerning 
the  upcoming  exercise.  One  problem  with  this  briefing  is  that  it  is,  in 
some  cases,  too  informative.  For  instance,  the  briefing  not  only  informs 
the  student  as  to  the  nature  of  a  simulated  malfunction  without  his  having 
to  test  for  it  but  the  correct  course  of  action  as  well. 

Adjust  Gunnery  Parameters.  There  are  a  few  special-purpose  features 
designed  to  adjust  parameters  of  tank  gunnery.  These  idiosyncratic 
features  were  given  names  that  are  descriptive  of  their  functions. 

1.  Ammunition  load.  Except  for  the  U-COFT,  the  simulators  have  the 
capability  to  vary  the  number  of  rounds  that  are  available  to  the  gunner. 
By  systematically  varying  the  load,  this  feature  allows  the  instructor  to 
train  the  gunner  and  tank  commander  to  be  more  aware  of  their  ammo  status 


and  to  adjust  their  behavior  accordingly.  Training  should  be  designed  to 
exploit  this  feature. 

2.  Variable  loading  interval.  VIGS  and  TopGun  have  the  capability 
to  vary  the  interval  from  the  fire  command  to  the  loader's  UP  announce¬ 
ment.  This  feature  prevents  the  gunner's  timing  from  becoming  too  rigid. 
There  are  at  least  two  possible  ways  that  this  feature  can  be  implemented. 
One  possible  strategy  is  to  decrease  the  loading  interval  as  the  student 
gunner  gains  skill.  This  strategy  simulates  the  improvement  of  loader 
performance  as  a  function  of  practice.  The  second  strategy  is  to  vary  the 
loading  times  according  to  a  random  distribution  that  is  representative  of 
loader  performance.  The  latter  strategy  should  train  the  gunner  to  aijust 
to  realistic  variations  in  loading  times.  In  either  case,  until  some  basic 
questions  about  the  loading  interval's  relationship  to  gunnery  training 
and  performance  are  answered,  the  appropriate  strategy  for  implementing 
this  instructional  feature  remains  uncertain. 

3.  Reticle  misalignment.  The  VIGS  has  the  capability  of  inducing  a 
first  round  error  by  misaligning  the  reticle  aiming  and  target  impact 
points  for  practicing  burst-on-fire  adjustment  techniques.  According  to 
this  technique,  the  gunner  compensates  for  a  first-round  miss  by  changing 
his  aim  according  to  the  perceived  relation  between  the  point  of  impact 
and  the  target.  Burst-on-target  was  the  preferred  method  of  fire 
adjustment  for  M60A1  tanks.  However,  with  the  advent  on  the  advanced  fire 
control  system  of  the  Ml,  the  burst-on-target  technique  was  superceded  by 
the  reengagement  method.  According  to  the  reengagement  method,  the  gunner 
enters  a  new  ballistic  solution  in  the  computer  and  relays  the  reticle  on 
the  target's  center  of  mass.  Given  reticle  misalignment,  the  gunner  could 
achieve  a  second  round  hit  using  the  burst-on-target  method  but  would 
continue  to  miss  using  the  reengagement  technique.  Therefore,  this 
feature  is  not  appropriate  for  training  the  newer  doctrine  of  fire 
adjustment. 

Simulate  Events.  Finally,  there  are  a  set  of  so-called 
"instructional  feature s"  that  enable  the  instructor  to  simulate  some  event 
or  events  that  are  relevant  to  training.  Because  they  relate  to  the 
simulation  rather  than  training  per  se,  they  are  more  appropriately  viewed 
as  fidelity  features  even  though  they  are  traditionally  classified  as 
instruct ional  features.  Furthermore,  the  criteria  for  effectiveness  is 
not  whether  or  not  these  features  increase  the  amount  or  rate  of  learning, 
but  rather  whether  or  not  they  are  required  to  simulate  task  events. 

1.  Automated  crewmembers.  If  a  particular  crewmember  cannot  be 
available  for  training,  his  inputs  and  outputs  must  somehow  be  simulated. 
For  instance,  the  intention  of  the  low-cost  simulators  (TopGun  and  VIGS) 
is  to  train  gunners  apart  from  other  armor  crewmen.  Thus,  input  from  the 
tank  commander  and  loader  must  be  simulated.  In  U-COFT,  the  tank 
commander  is  trained  with  the  gunner;  therefore,  he  does  not  have  to  be 
simulated,  but  the  loader's  input  and  output  must  be  simulated  (see 
below).  Finally,  SIMNET  requires  all  four  crewmen;  thus,  crewmembers  do 
not  have  to  be  automated. 


2.  Malfunction  selection.  In  the  gunnery  domain,  malfunction 
selection  refers  to  the  capability  of  simulating  degraded  gunnery  modes. 

Of  the  four  devices,  the  U-COFT  is  the  only  one  to  offer  practice  on  a 
variety  of  degraded  modes.  (See  previous  section  for  details.)  The  only 
other  device  that  simulates  a  malfunction  is  TopGun,  which  can  present  the 
GAS  reticle  to  simulate  a  GPS  malfunction. 
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3.  Ammo  select/reload.  Upon  hearing  the  tank  commander's  fire 
command,  the  U-COFT  instructor  presses  the  ammo  select  or  reload  button  to 
initiate  the  simulation  of  the  sound's  and  time  intervals  associated  with 
loading  the  round.  Most  importantly,  the  ammo  select/reload  buttons 
select  the  ballistic  trajectory  simulations  associated  with  the  selected 
ammunition.  These  keys  are  not  required  for  the  low-cost  simulators  (VIGS 
and  TopGun)  wherein  the  ammunition  to  be  fired  is  programmed  to  match  the 
fire  command.  Nor  is  it  required  in  SIMNET  where  the  loader  initiates  the 
simulation  by  pushing  the  appropriate  buttons  on  the  ready  rack. 

Conclusions 

In  this  section,  we  summarize  the  results  of  the  evaluation  by 
describing  the  usefulness  of  instructional  features  to  the  proposed 
project.  As  in  the  previous  section,  the  discussion  is  organized  around 
the  basic  instructor  functions  that  instructional  features  support. 

Prepare  Training.  The  device-based  training  program  will  require 
instructors  to  select  training  exercises  according  to  certain  parameters. 
All  devices  have  some  capability  in  this  regard,  but  SIMNET  appears  more 
cumbersome  to  use.  The  ability  to  sequence  training  is  less  crucial  to 
the  project. 

Control  Practice.  In  the  device-based  training  program,  the  instruc¬ 
tor  should  be  able  to  control  practice  closely  in  terms  of  providing 
feedback  and  repetitive  training.  For  those  functions,  U-COFT  and  TopGun 
appear  to  be  superior  to  both  SIMNET  and  VIGS. 

Administer  and  Monitor  Training.  The  training  administration 
features  (automated  training  management  and  briefing)  implemented  in  the 
U-COFT  have  no  applicability  to  the  current  project.  However,  the  U- 
COFT's  ability  to  closely  monitor  gunnery  performance  is  a  distinct 
advantage.  The  latter  capability  is  discussed  more  fully  in  the  next 
section . 
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Adjust  Gunnery  Parameters.  As  discussed  earlier,  the  ammo  load  and 
loading  interval  features  offer  potential  training  benefits.  Questions 
remain,  however,  concerning  how  these  features  may  be  effectively  imple¬ 
mented.  The  device-based  training  program  will  not  exploit  these  features 
until  the  questions  are  answered. 


CHAPTER  6 


IDENTIFICATION  AND  EVALUATION  OF  DEVICE  FEATURES 
REQUIRED  FOR  TESTING 


This  chapter  extends  our  device  evaluations  to  testing  applications. 
In  the  first  portion  of  the  chapter,  a  number  of  testing  requirements  are 
discussed  and  our  evaluation  questions  presented.  In  the  last  portion, 
device  evaluations  are  presented.  As  with  the  training  evaluations,  our 
conclusions  are  rational  analyses  of  device  capabilities  as  opposed  to 
empirical  data  on  student  performance.  In  essence,  they  are  hypotheses 
about  what  should  be  valid  testing  approaches  and  caveats  about  uses  that 
may  best  be  avoided  until  empirical  evidence  is  available. 


Device  Testing  Requirements 


In  Chapter  4,  we  discussed  a  number  of  issues  regarding  testing 
requirements.  In  that  discussion,  we  focused  on  issues  created  by  the 
nature  of  what  was  being  measured-individual  performance  in  the  crew 
context  of  Ml  tank  tactical  gunnery.  In  this  chapter,  we  focus  on  issues 
created  by  the  nature  of  the  testing  medium. 

Technical  considerations  for  assessing  individual  differences  are 
most  often  discussed  in  the  context  of  achievement,  aptitude,  and 
personality  testing  emphasizing  psychometric  characteristics  associated 
with  the  reliability  and  validity  of  the  measurement  instrument. 
Reliability  and  validity  remain  important  requirements  for  simulators  used 
to  assess  performance  during  the  acquisition  and  sustainment  of  gunnery 
skills.  Furthermore,  with  the  tremendous  and  varied  potential  for 
measures  and  scores  from  sophisticated  simulators,  it  is  equally  important 
to  consider  information  utility,  including  questions  about  data  capture 
and  storage,  real-time  display  of  performance  information,  and  flexibility 
of  problem  presentation. 


Validity 

The  traditional  approach  for  evaluating  the  validity  of  job  sample 
measures  of  performance  has  been  content  validity  (Cronbach,  1960; 
Wernimont  &  Campbell,  1969).  The  implicit,  underlying  assumption  is  that 
the  performance  requirements  of  the  test  are  equivalent  to  the  performance 
requirements  of  the  job.  For  content  validity,  attention  focuses  on  the 
extent  to  which  all  job  requirements  are  covered  by  the  test  and  on  the 
methods  use  to  score  test  performance  (Guion,  1977;  Hambleton,  1980).  For 
simulators,  concern  over  fidelity  creates  more  uncertainty  about  the 
equivalence  of  performance  requirements.  Therefore,  questions  concerning 
the  concurrent  or  predictive  validity  of  the  test  are  also  evoked.  Our 
evaluation,  however,  is  necessarily  limited  to  fidelity,  domain  coverage, 


and  scoring  procedures  because  we  have  very  limited  performance  data  on 
which  to  base  concurrent  or  predictive  validity  judgments. 

Domain  Coverage.  The  analysis  of  the  Ml  gunnery  domain  was  con¬ 
structed  to  include  every  important  decision  and  action  in  the  context  of 
tactical  gunnery.  No  superfluous  behaviors  were  included.  Furthermore, 
the  tactical  gunnery  domain  has  already  been  partition  into  activities  and 
options  within  those  activities.  Thus,  it  is  important  to  make  sure  that 
every  activity  and  every  element  in  every  activity  is  learned.  This  also 
means  that  there  should  be  some  means  of  assessing  performance  to  provide 
appropriate  feedback  for  every  element.  Furthermore,  we  have  argued  in 
Chapter  4  that  it  is  most  appropriate  to  test  integrated  chunks  of  the 
procedure,  such  as  our  activities  or  objectives.  Consequently,  it  is  not 
appropriate  to  test  chunks  that  have  missing  elements.  There  are  three 
questions  about  element  coverage  that  much  be  evaluated  in  order  to 
determine  whether  an  activity  (or  an  option  within  an  activity)  is 
testable. 

1.  Does  the  device  present  the  opportunity  to  perform  each  element? 

2.  Are  the  element  measurement  requirements  (knowledge,  behavior, 
and/or  outcome)  automatically  recorded  for  the  elements? 

3.  Can  an  instructor  observe  and  score  the  elements  in  an 
appropriate  measurement  mode? 

The  performance  model  and  the  measurement  mode  specification  rules, 
presented  in  the  previous  chapter,  prescribe  what  constitutes  appropriate 
means  for  assessing  each  gunnery  task  element. 

A  second  aspect  of  domain  coverage  concerns  coverage  of  the  gunnery 
conditions.  This  evaluation  is  described  in  Chapter  5  and  presented  in 
Appendix  F.  Because  of  the  built-in  redundancy  between  the  activity 
designations  and  the  conditions,  evaluation  of  whether  primary  conditions 
(e.g.  type  of  target,  equipment  status)  are  presented  are  implicit  in  our 
domain  coverage  evaluations.  Secondary  conditions  (e.g.  NBC  conditions, 
target  sector)  are  evaluated  only  in  Appendix  F.  For  any  activity,  the 
more  of  these  secondary  conditions  that  can  be  represented  by  a  device, 
the  less  deficient  the  evaluation.  For  example,  from  the  Grafenwohr  data 
presented  in  Chapter  3,  we  know  that  NBC  conditions  produce  lower  gunnery 
performance.  We  do  not  know  whether  this  decrement  is  the  same  for  all 
crews.  Therefore  inferring  non-NBC  performance  from  NBC  performance  may 
be  erroneous.  Testing  under  both  conditions  is  more  appropriate. 

Element  Scoring  Criteria.  While  our  prescriptions  for  measurement 
mode  provide  a  starting  point  for  evaluating  scoring  criteria,  there  is 
still  concern  for  the  relationship  between  the  score  and  the  dimension 
that  underlies  the  score. 


One  concern  is  whether  the  scale  used  to  quantify  performance 
provides  a  complete,  accurate  portrayal  of  the  performance.  Inappropriate 
scales  or  unnecessary  categorizations  can  distort  information  about 
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performance.  For  example,  if  a  gunner's  lay  at  the  time  of  firing  is 
simply  scored  as  on  or  off  target  rather  than  as  lay  error,  information  is 
lost  and  the  power  to  discriminate  among  various  levels  of  performance  is 
reduced.  Similarly,  scales  may  be  unrecognized  combinations  of  perform¬ 
ance  elements.  "Lay  error"  may  refer  to  sight  picture  or  the  fall  of  the 
round.  If  it  refers  to  fall  of  the  round,  it  is  not  a  truthful  index  of 
gunner's  lay,  but  is  contaminated  by  switch  settings,  ranging,  and  perhaps 
even  round-to-round  dispersion. 

A  second  problem  is  the  likelihood  of  correct  performance.  It  is  a 
joint  function  of  simulator  design,  scoring  design,  and  the  nature  of  the 
behavior.  Although  it  may  be  appropriate  to  cue  and  shape  behavior  during 
training  by  highlighting  correct  controls,  switches,  or  perceptual  cues, 
score  interpretation  must  be  made  in  light  of  such  alterations. 

Composite  Scores.  As  discussed  in  Chapter  4,  composite  scores  may  be 
viewed  as  abstractions  expressing  the  utility  of  performance  for  some 
particular  purpose.  Two  different  purposes  were  identified:  (a)  utility 
of  expected  "real  world"  outcomes  of  performance,  and  (b)  utility  of  the 
performance  trial  for  skill  acquisition.  There  is  no  a  priori  reason  to 
believe  that  a  single  composite  score,  produced  by  a  single  utility 
function,  will  be  appropriate  for  both  uses.  Therefore  to  evaluate  any 
composite  score  produced  by  a  gunnery  simulator,  we  need  to  ask  a  number 
of  questions.  Is  the  purpose  of  the  composite  clear,  and  does  its 
construction  (the  rules  or  formulas)  support  the  purpose?  If  the  purpose 
of  a  composite  is  not  made  explicit,  the  questions  are  reversed.  What  are 
of  the  rules  underlying  the  composite,  and  what  use(s)  do  such  rules  seems 
to  support?  What  meaning  do  the  scores  have  in  terms  of  expected 
performance  in  future  training?  What  meaning  do  the  scores  have  in  terms 
of  expected  performance  in  the  "real  world?" 


Rel  iabi 1  ity 

Measurement  reliability  is  a  function  of  individual  variability  in 
observed  behavior  in  relation  to  the  variability  across  individuals.  The 
more  consistent  each  individual's  scores  in  comparison  to  the  variance 
across  persons,  the  greater  the  reliability  of  the  measure.  Several 
fairly  distinct  factors  influence  the  variability  of  individual  scores. 
The  heterogeneity  of  the  domain  being  measured,  the  level  of  skill 
proficiency,  and  the  resulting  stability  of  performance  are  inherent 
features  of  the  task  that  cannot  be  directly  controlled  by  testing 
instrument  design.  However,  the  extent  to  which  a  device  supports 
selection  and  repetition  of  engagement  sequences  to  provide  multiple 
observations  of  performance  is  an  important  feature  for  overcoming 
inherent  performance  variability.  Other  factors  are  more  closely 
associated  with  device  design.  These  include  measurement  instrument 
characteristics,  including  equivalence  between  different  devices  of  the 
same  type,  precision  of  the  scoring  criteria,  and  the  extent  to  which 
judgments  are  eliminated  in  the  measurement  process. 


Performance  Stability.  A^art  from  any  device  or  testing  medium 
considerations,  reliability  is  directly  related  to  variability  in 
performance  over  repetitions  of  the  same  problem.  Particularly  in  the 
early  stages  of  skill  acquisition,  performance  is  not  perfectly  correlated 
from  one  repetition  to  the  next  (Jones,  Kennedy,  &  Bittner,  1981).  The 
person  is  changing  and  a  plot  of  performance  scores  is  apt  to  be  a  jagged, 
but  rising  curve.  The  trend  in  the  curve  is  only  apparent  from  visual 
smoothing  of  the  peaks  and  valleys.  Depending  on  the  amplitude  of  the 
peaks  and  valleys,  no  one  point  is  very  descriptive  of  such  a  curve. 
Likewise,  depending  on  the  extremes  of  trial  to  trial  variation,  no  one 
measurement  is  very  descriptive  of  an  individual's  true  performance 
ability.  Therefore,  repetition  of  measurement  through  repetition  of 
problem  presentation  is  an  important  testing  characteristic  for  gunnery 
simulators. 

Measurement  Instrument  Consistency.  Variations  in  presentation  of 
stimulus  or  sensitivity  of  control  manipulation  across  repetitions  for  the 
same  machine,  or  between  machines  can  create  extraneous  variance  in 
performance  scores  that  reduces  reliability.  For  example,  all  the  of 
devices  considered  in  this  report  present  visual  stimuli  via  a  color 
monitor.  Differences  between  units  of  the  same  type  in  color  and  contrast 
adjustment  can  make  targets  harder  to  detect  in  one  unit  compared  to  the 
other.  Computer  driven  devices  of  the  type  being  used  for  Ml  gunnery 
simulation  can  be  also  sensitive  to  adjustments  in  electrical  and  mechani¬ 
cal  controls  (e.g.  power  control  handles,  driver's  T-bar)  linked  with 
computer  processing  units. 

A  second  aspect  of  machine  consistency  is  that  several  of  the  devices 
we  are  examining  are  either  prototypes  or  in  the  early  stages  of 
production.  At  times  model  changes  are  introduced  as  new  units  are 
delivered.  Thus,  it  cannot  be  assumed  that  two  apparently  similar  units 
of  the  same  type  are  in  fact  the  same.  Thus,  in  order  to  evaluate  test 
consistency,  we  must  determine  where  model  changes  have  created 
differences  among  units  of  the  same  device. 

Elimination  of  Judgment.  A  final  consideration  for  reliability  is 
the  extent  to  which  the  device  eliminates  the  need  for  a  scorer  to  monitor 
and  evaluate  performance.  Although  automatic  scoring  has  other 
considerations  (see  discussion  on  validity  above),  certainly  eliminating 
human  observation  and  judgment  can  increase  consistency  of  measurement 
(Boldovici,  Osborn,  &  Harris,  1977). 


Information  Utility 

Aside  from  questions  of  reliability  and  validity,  performance 
information  must  be  delivered  to  gunnery  instructors  and  students  in  a 
usable  and  timely  manner. 

Real-time  Presentation  of  Performance  Information.  We  have  presented 
tactical  gunnery  as  a  complex  procedure  with  few  complex  motor  skill 
requirements.  Diagnostic  feedback,  therefore,  should  focus  primarily  on 


whether  specific  actions  took  place,  with  additional  information  about  how 
well  the  behavior  was  executed  for  some  elements.  Because  there  may  be  up 
to  four  persons  interacting  as  a  crew  in  a  training  session,  presentation 
of  information  about  all  elements  would  be  too  fast  for  a  single  instruc¬ 
tor  to  handle.  The  most  obvious  solution  is  to  present  information  only 
about  omissions  and  deficiencies  in  performance.  Certainly,  consideration 
needs  to  be  given  about  when  feedback  is  best  given  (during  execution  or 
before  the  next  trial)  with  performance  information  displays  timed 
accordingly. 

In  addition  to  amount  and  timing  of  information,  presentation  medium 
is  also  an  important  consideration.  Check  lists,  numbers,  pictures, 
graphs,  and  narratives  are  all  options  for  presenting  information  about 
performance.  Video  recordings  can  present  a  real-time  re-creation  of 
displays  viewed  by  crew  members  during  their  performance  trial.  Our  rule 
of  thumb  for  presentation  medium  is  that  information  displays  should 
support  the  data  requirements  specified  by  our  knowledge,  behavior,  or 
outcome  measurement  mode  assignments. 

Data  Capture  and  Storage.  Computer  driven  simulators  also  offer  a 
tremendous  capacity  to  capture  and  store  performance  data.  Differences 
inherent  in  hardware  capabilities  suggest  two  kinds  of  storage  and 
retention  intervals.  We  make  a  distinction  between  short-  and  long-term 
storage  with  short-term  storage  being  automatically  erased  or  replaced 
with  the  start  of  each  new  exercise,  and  long-term  storage  retained  until 
erased  by  an  operator.  The  former  can  be  used  for  immediate  feedback 
while  the  later  can  be  used  to  review  performance  over  several  sessions. 
Long-term  storage  can  also  be  used  for  research  purposes  comoaring 
difference  types  of  persons  or  intercorrelating  performance  on  different 
activities  or  different  conditions.  Patterns  of  errors  can  be  detected 
and  perhaps  instructions  geared  toward  preventing  them  can  be  prepared  and 
tested.  Analysis  of  individual  differences  in  acquisition  curves  may  lead 
to  the  capacity  to  predict  terminal  performance  from  performance  in  early 
trials.  Of  course,  the  usefulness  of  data  capturing  and  storage  is 
limited  by  the  quality  of  the  data  collected  as  determined  by  the 
considerations  of  reliability  and  validity. 

Adaptive  or  Flexible  Problem  Presentation.  Testing  (as  well  as 
training)  efficiency  is  increased  to  the  extent  that  the  testing  medium 
can  be  directed  to  the  specific  objectives  of  most  concern.  This  is  the 
underlying  premise  of  adaptive  testing.  These  are  several  levels  at  which 
a  device  may  support  adaptive  testing.  At  a  minimum  the  device  should 
allow  the  operator  to  select  parameter  conditions  in  order  to  select 
problems  of  a  specific  type.  At  this  level,  selection  is  left  to  the 
operator  guided  by  whatever  testing  protocols  are  available.  Alterna¬ 
tively,  a  device  may  be  pre-programmed  to  select  problems  based  on  each 
student's  performance.  An  adaptive  test,  either  operator  or  device 
controlled,  also  requires  that  gunnery  problems,  defined  by  variations  in 
condition  presentation,  be  hierarchically  ordered  to  the  extent  possible. 
The  present  report  presents  hierarchical  analyses  of  gunnery  skills  but 
they  have  not  been  empirically  verified.  A  more  sophisticated  device  may 
be  designed  to  branch  individual  students  based  on  performance,  to  record 


patterns  of  performance  across  individuals,  and  to  make  adjustments  in  its 
branching  routine.  In  the  latter  case,  the  device  is  acting  as  both  a 
testing  device  and  an  automated  research  tool.  None  of  the  devices  we 
reviewed  approach  this  level  of  sophistication. 


Device  Evaluation  Questions 

The  above  array  of  evaluation  issues  includes  questions  at  the 
behavior  level  and  at  the  activity  or  training  objective  level.  At  the 
element  level  are  questions  about  the  presence  of  the  element,  availa¬ 
bility  of  automatic  scoring,  scoring  criteria,  data  presentation  medium, 
and  storage.  For  elements  not  automatically  scored,  there  is  the  possi¬ 
bility  of  scoring  by  an  instructor.  At  the  activity  level  are  issues 
related  to  real-time  data  presentation  load  and  timing,  repetition  of 
performance  trials,  instrument  consistency  and  composite  scores.  These 
issues  are  not  independent;  however,  a  concise  set  of  device  evaluation 
questions  were  derived.  They  are  presented  in  Table  6-1. 

The  questions  are  logically  divided  into  element  level  and  activity 
level  concerns.  The  first  element  question,  "Can  the  element  be 
performed?"  has  already  been  covered  in  the  evaluation  of  fidelity 
presented  above.  For  each  of  the  activity  divisions,  a  determination  then 
is  made  as  to  the  adequacy  of  the  elements  present  for  representing  the 
activity.  This  is  a  subjective  judgment  based  on  the  number  and 
criticality  of  missing  elements.  It  may  only  take  a  few  missing  or 
inadequately  represented  elements  to  invalidate  testing  of  a  particular 
activity.  If  an  activity  cannot  be  tested,  then  testing  of  individual 
elements  within  that  activity  that  are  represented  is  deficient,  although 
occasionally  useful.  That  is,  testing  isolated  elements  may  determine 
whether  the  behavior  can  be  executed,  but  does  not  determine  whether  it 
will  be  executed  as  part  of  the  activity  sequence.  Testing  isolated 
elements  makes  sense  for  high  skill  behaviors  and  behaviors  with  some 
underlying  knowledge  (decision)  requirement.  Target  tracking  is  certainly 
an  element  in  the  Ml  tactical  gunnery  domain  that  falls  into  this  latter 
category. 


Evaluation  of  Devices 


Procedure 

The  complete  tactical  gunnery  domain  listing  of  tank  commander, 
gunner,  driver  and  loader  elements  was  converted  to  separate  check  lists 
for  each  position.  For  each  element,  the  measurement  mode  was  specified 
according  to  the  scheme  presented  in  Chapter  4,  Tables  4-1  and  4-2.  These 
check  lists  were  used  to  rate  device  representation  and  scoring  for  each 
gunnery  element.  The  key  to  the  rating  procedure  is  presented  in 
Table  6-2.  Based  on  these  element  level  ratings  and  information  about 
device  operation  and  scoring  routines,  device  testing  capabilities  were 


Table  6-1 


Evaluation  Questions  Concerning  the  Use  of  Simulators  as  Testing  Devices 


Element  Level  Questions 


Activity  Level  Questions 


I.  Can  the  element  be  performed?  If 
not,  the  remaining  questions  are 
not  relevant. 


II.  Is  the  desired  measurement  mode(s) 
(K,B,  or  0)  automatically  assessed 
and  available  for  feedback? 

A.  Is  the  scoring  criteria 
appropriate? 

B.  Is  the  timing  and  medium  of 
presentation  of  the 

i  formation  appropriate? 

C.  Is  the  (element  level)  data 
stored  for  end-of-trial  term 
use? 

D.  Is  the  (element  level)  data 
stored  for  long  term  use? 

III.  Can  an  instructor  score  the 
element? 

A.  If  a  behavior  measure  is 
needed,  can  an  instructor 
observe  the  behavior? 

B.  If  outcome  measure  is  needed, 
can  an  instructor  observe  the 
outcome  of  the  behavior  either 
directly  or  by  some  remote 
display? 

C.  For  either  measure,  are  there 
or  can  there  be  clear  and 
distinct  guidelines  for 
scoring. 


1.  Is  the  domain  covered 
sufficiently  to  construct  a 
meaningful  test?  If  not, 
testing  is  inappropriate  and 
the  remaining  questions  are  not 
relevant. 

2.  Is  a  composite  score 
constructed? 

a.  Is  its  meaning  comprehens¬ 
ible? 

b.  Does  it  emphasize 
performance  utility  or 
training  utility? 

c.  Is  the  (composite  level) 
data  stored  for  end-of- 
trial  term  use? 

d.  Is  the  (composite  level) 
data  stored  for  long  term 
use? 

3.  Are  there  instrument 
inconsistencies? 

4.  Is  selection  of  engagement 
trials  possible? 

5.  Is  repetition  of  engagement 
trials  possible? 


'm  1 


Table  6-2 


Ratings  Used  for  Element  Level  Descriptions  of  Oevice  Testing  Capabilities 


Column  Headinq 

Symbol  and  Interpretation 

Measure  Required 

K  -  Knowledge  assessment 

B  -  Behavior  assessment 

0  -  Outcome  assessment 

,  -  Used  to  join  optional  assessment  modes  (e.g.  K,B  -  use  knowledge  or  behavior 

assessment). 

K&B  -  Knowledge  and  behavior  both  required  for  diagnostic  assessment. 

(0)  -  Outcome  assessment  can  be  used  for  proficiency  assessment,  but  has  limited 
diagnostic  value. 

Element  Represented 

Y  -  Performance  of  the  element  is  expected  to  mirror  performance  on  the  actual 
equipment. 

(Y)  -  Performance  of  the  element  is  sufficiently  different  from  the  actual  equipment 
to  reduce  expected  validity. 

Partial  -  Some  portion  of  the  performance  requirements  are  not  represented. 

Degraded  -  degraded  stimulus  conditions  are  expected  to  alter  perceptual  requirements 
for  the  element. 

N  -  Element  can  not  be  performed  sufficiently  to  score  performance. 

Automatic  Recording 

Mode 

B  -  Behavior  is  recorded  or  scored 

0  -  Outcome  of  the  element  is  recorded  or  scored 

T  -  Elapsed  time  to  element  performance  is  recorded 

[Note  -  behaviors  or  outcomes  of  elements  that  are  remotely  displayed  at  the  time  of 
execution,  but  not  captured  for  later  review  are  not  counted  in  this  column] 

Automatic  Recording 

Feedback 

Display  -  Occurrence  of  the  behavior  or  evaluation  of  the  outcome  is  printed  or 
displayed  on  a  monitor. 

Replay  -  Occurrence  of  the  behavior  or  representation  of  the  outcome  of  the  element 
can  be  watched  on  a  video  replay  of  the  performance. 

Automatic  Recording 

Storage 

L  -  Behavior  or  outcome  records  are  "permanent"  in  the  sense  that  they  are  not 
automatically  erased/lost  when  the  exercise  or  training  session  is  over. 

S  -  Behavior  or  outcome  records  are  displayed  at  the  end  of  the  exercise  and  then 
erased/ lost. 

Instructor  Scoring 

K  -  Instructor  can  observe  performance  or  a  remote  display  of  performance  and 
score  knowledge. 

B  -  Instructor  can  observe  performance  or  a  remote  display  of  performance  and 
score  behavior. 

0  -  Instructor  can  observe  performance  or  a  remote  display  of  performance  and 
score  outcome. 

Element  Testable 

Y  -  Element  can  be  scored  either  automatically  or  by  an  Instructor. 

(Y)  -  Element  can  be  scored,  but  reduced  validity  is  expected. 

N  -  Element  can  not  be  scored. 

then  rated  at  the  summary  level  for  each  of  the  activities  and  options 
within  the  activities.  The  key  for  the  summary  level  descriptions  is 
presented  in  Table  6-3. 
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Table  6-3 


Ratings  Used  for  Activity  Level  Descriptions  of  Device  Testing 
Capabilities 


Column  Head  in 


Symbol  and  Interpretation 


Domain  Testable 

Y  - 

Elements  are  sufficiently  represented  to  expect 
valid  information. 

(V)  - 

Elements  representation  is  somewhat  degraded 
and  information  may  have  marginal  validity. 

N  - 

Element  representation  is  sufficiently  degraded 
to  expect  little  test  validity. 

Composite  Meaningful 

Gives  a  short  description  of  any  activity  level 
scores  provided. 

Blank  - 

No  composite  score  is  computed  for  the  domain. 

Composite  Storage 

Gives  a  short  description  of  activity  level 
scores  storage  for  long  or  short  term  access. 

Selection/Repetition 

Describes  device  capabilities  for  selecting 
exercises,  e.g.  programming  a  particular 
sequence  of  engagements. 

Device 

Inconsistencies 

Notes  any  characteristics  that  vary  among 
devices  and  could  affect  performance  scores. 

Subjective  Ratings 


Ratings  of  "Element  Represented"  and  "Domain  Testable"  are  the  two 
primary  evaluation  points.  Both  are  subjective.  Element  represented 
ratings  are  based  on  the  fidelity  ratings  of  how  closely  each  element  of 
the  simulated  task  corresponds  to  the  actual  task.  These  ratings  were 
presented  in  the  previous  chapter. 


LV* 


An  underlying  assumption  is  that  element  correspondence  must  be 
greater  for  testing  than  for  training.  That  is,  some  reduction  in  element 
fidelity  can  occur  and  still  allow  some  practice  of  task  requirements 
within  an  activity.  That  practice  might  be  expected  to  transfer  to 
performance  on  the  actual  equipment  if  some  time  is  allotted  on  the  actual 
equipment  to  make  adjustments.  For  testing,  the  difference  in  performance 
requirement  between  the  device  and  the  real  equipment  constitutes  an  extra 
component  in  a  device  score  that  has  no  analogous  component  in  the  actual 
equipment.  The  rule  of  thumb  is  that  if  there  is  an  extra  requirement  of 
the  device  needed  to  allow  practice,  than  the  validity  of  testing  in  that 
domain  is  reduced.  Our  tendency  is  to  be  conservative  in  evaluating 
device  capabilities  for  providing  performance  scores  that  can  readily  be 
translated  into  proficiency  on  the  actual  equipment. 


General  Conclusions 


Ratings  made  to  evaluate  device  testing  capabilities  at  the  element 
level  and  at  the  activity  level  are  presented  in  Appendix  I.  Summary 
evaluations  of  device  capabilities  for  testing  are  presented  in 
Figures  6-1  to  6-3.  Figure  6-1  evaluates  each  of  the  four  devices  for 
testing  gunners.  Figure  6-2  evaluates  U-COFT  and  SIMNET  for  testing  tank 
commanders.  Figure  6-3  evaluates  SIMNET  for  testing  drivers  and  loaders. 


Despite  plans  to  the  contrary,  no  distinction  between  proficiency 
testing  and  diagnostic  testing  was  made  in  the  evaluations.  Our  gunnery 
domain  analysis  is  comprehensive  and  detailed.  As  a  result,  our 
expectations  for  diagnostic  testing  are  great.  Complete  diagnostic 
testing  of  performance  deficiencies  requires  that  scores  be  available  for 
every  element.  For  all  devices,  automated  scores  were  limited  in  number 
and  tended  to  be  outcome  oriented.  Outcome  measures  can  suffice  for 
diagnostic  applications  only  for  the  most  simple  elements  where  the 
procedure  is  strictly  linear  and  there  are  no  skilled  behaviors  (Type  1 
elements  as  described  in  Chapter  4).  Thus,  diagnostic  testing  with  any  of 
the  devices  would  require  instructor  support  to  provide  element  level 
evaluations.  Given  reliance  on  instructor  support,  the  key  evaluation 
criteria  for  both  proficiency  testing  and  diagnostic  testing  is  sufficient 
domain  coverage.  Furthermore,  from  the  discussion  in  Chapter  4,  near 
perfect  presentation  of  the  elements  is  required  to  obtain  complete 
diagnostic  information  or  to  obtain  a  valid  domain  score.  Thus,  our 
device  evaluations  for  testing  were  influenced  most  heavily  by  element 
fidelity  and  domain  coverage.  The  ratings  that  appear  in  Figures  6-1  to 
6-3  indicate  our  assessment  of  the  capability  of  a  device  to  provide  a 
valid  proficiency  and/or  diagnostic  assessment  of  skills  within  each 
division  of  the  domain  on  the  assumption  that  assessment  will  be  supported 
by  an  instructor  participating  in  the  scoring  process. 


i 


Given  the  emphasis  on  device  fidelity,  it  is  not  surprising  that  our 
conclusions  for  testing  capabilities  parallel  our  conclusion  for  training 
capabilities.  That  is,  VIGS  and  TopGun  provide  gunner  marksmanship 
training  only,  so  their  testing  potential  is  limited  to  gunnery  domain 
Activities  4  through  7.  Even  for  these  activities,  their  coverage  is 
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Figure  6-2.  Evaluation  of  device  utility  for  testing  tank  commander  performance. 
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Figure  6-3.  Evaluation  ot  SlttNET  utility  for  testing  driver  and  loader  performance  (cont'd). 


incomplete.  While  this  does  not  diminish  their  potential  importance  as 
devices  for  practicing  tracking  skills,  they  are  not  strongly  recommended 
for  testing  in  these  areas.  U-COFT,  again  not  unexpectedly,  captures  the 
highest  recommendations  for  testing  use.  By  including  a  TC  station  and 
degraded  mode  capabilities,  a  significantly  greater  portion  of  the  gunnery 
domain  is  covered,  and  it  appears  to  have  potential  to  provide  valid 
scores  for  gunner  and  TC  skills.  SIMNET  has  some  design  limitations 
(e.g.,  no  night  simulation,  TIS,  Coax  or  Cal  .50)  that  reduce  its  activity 
domain  coverage,  and  its  configuration  (no  external  monitor  dedicated  to 
observing  gunner's  or  TC's  sight  picture)  reduces  scoring  possibilities. 
Its  greatest  potential  appears  to  be  for  TC  testing  in  Activities  4,  5,  7 
and  11.  SIMNET  driver  and  loader  stations  offer  little  potential  for 
testing  skills  of  these  two  crew  members. 

Examination  of  the  conditions  evaluations  in  Appendix  F  also  shows 
the  superiority  of  UCOFT  and  SIMNET.  VIGS  and  TopGun  provide  only  the 
basic  variety  of  conditions  needed  to  support  marksmanship  training. 
SIMNET,  with  its  free  play  format,  provides  potentially  the  greatest 


variety  of  conditions,  particularly  with  the  mission  requirements 
parameters  (13  through  18).  After  considering  these  evaluations  in 
relation  to  the  element  fidelity  and  domain  coverage  evaluations,  our 
conclusion  was  that  variations  in  the  representation  of  secondary 
conditions  was  not  sufficient  to  alter  our  appraisals  of  device  validity. 
Therefore,  the  appraisal  of  secondary  conditions  carried  little  weight 
over  the  domain  coverage  analysis. 


Automated  Feedback  Capabilities 

Each  of  the  devices  does  present  some  type  of  feedback.  However, 
feedback  from  none  of  the  devices  matches  up  with  our  measurement 
specifications  very  closely  (see  the  "Automatic  Recording"  ratings  in 
Appendix  I).  The  data  that  is  provided  is  mixed.  Part  of  the  information 
may  be  rather  "raw"  (e.g.  what  ammunition  was  indexed  rather  than  whether 
the  correct  ammunition  was  indexed)  and  other  highly  processed  (e.g. 
engagement  "points"  based  on  some  combination  of  rounds  fired,  time  and 
target  hit,  with  a  system  dispersion  factor  added  in).  The  emphasis  is  on 
speed  and  target  hits  with  little  attempt  to  tease  out  the  factors  that 
influence  hits  (e.g.,  sight  picture,  range  in  computer,  smoothness  of 
tracking  and  lead  solution,  ammunition  indexed,  and  system  error). 
Curiously,  there  is  a  tendency  to  contaminate  hit  scores  with  round-to- 
round  dispersion.  Certainly,  it  is  appropriate  to  give  crews  the 
experience  of  rounds  missing  even  when  they  do  everything  correctly.  On 
the  other  hand,  scoring  and  feedback  should  clearly  indicate  that  the  crew 
indeed  performed  correctly.  TopGun  appears  to  do  this.  When  a  round 
misses  due  only  to  a  system  dispersion  factor,  lay  error  registers  zero. 

Nowhere  in  our  specification  of  element  level  measures  did  we 
indicate  that  target  hit  should  be  used  as  an  outcome  measure.  Target  hit 
is  a  composite  measure  of  crew  proficiency.  Lay  error  is  provided  by  U- 
COFT,  VIGS  and  TopGun,  but  in  each  case  it  appears  to  be  in  relation  to 
the  projected  fall  of  the  round.  This  index  is  also  a  composite  score. 


It  is  a  product  of  ammunition  indexed,  ammunition  loaded,  correct  ranging, 
and  gunners  tracking.  Any  one  of  these  could  cause  a  "lay  error." 
Furthermore,  if  an  incorrect  range  is  entered,  or  the  ammunition  selection 
switch  setting  does  not  match  the  ammunition  loaded,  then  gunner's 
tracking  or  lay  cannot  be  evaluated.  Our  argument  is  not  against  hits  as 
a  crew  outcome  criterion  measure.  Rather,  it  is  for  recognition  of  the 
fact  that  number  of  hits  is  a  score  which  synthesizes  several  performance 
elements,  making  diagnosis  of  any  single  performance  element  uncertain. 

Aside  from  miss-hit  information,  SIMNET  provides  no  automated 
feedback  on  gunnery  performance.  The  remaining  three  devices  in  this 
respect  are  described  below. 

U-COFT.  U-COFT  provides  the  most  extensive  feedback.  Gunnery 
performance  is  divided  into  three  categories:  target  acquisition,  reticle 
aim,  and  system  management.  These  categories  are  used  to  define  a  three 
dimensional  matrix  of  exercises  that  systematically  vary  in  difficulty. 
They  are  also  used  to  categorize  criteria  to  assess  performance  on  each 
engagement  within  the  exercises  (there  are  ten  targets  in  each  exercise, 
five  to  ten  engagements  depending  on  whether  they  are  single  or  multiple 
target  engagements).  Thus,  a  complete  description  of  TC  and  Gunner 
proficiency  is  a  combination  of  exercise  score  plus  level  of  the  exercise 
in  the  matrix.  Training  progress  through  the  matrix  is  based  on  exercise 
scores.  Table  6-4  presents  the  three  dimensions  and  the  criteria  used  to 
assess  performance  within  each  dimension. 

Certainly  the  U-COFT  feedback  is  useful.  In  addition  to  letter 
scores  for  each  of  the  three  dimensions,  "raw"  data  (e.g.  time,  number, 
and  type  of  switch  errors)  is  printed  in  a  summary  for  all  of  the 
engagements  in  an  exercise.  In  addition,  there  is  a  dynamic  situation 
monitor  that  shows  switch  setting  and  computer  entries,  including  range. 
The  situation  monitor  also  shows  a  history  of  lay  errors  for  the  last 
round  fired  at  each  target.  These  lay  errors,  as  well  as  the  lay  errors 
for  all  rounds  depicted  in  a  shot  pattern  print-out,  are  with  respect  to 
projected  round  impact.  They  are  not  a  straightforward  indication  of 
sight  picture  unless  target  range  in  entered  perfectly  and  there  are  no 
switch  errors.  While  "lay  error"  history  and  switch  errors  is  retained 
and  can  be  printed  out,  range  entered  at  the  time  of  firing  is  not 
automatically  captured.  Thus,  the  U-COFT  instructor/operator  (I/O)  must 
watch  the  situation  monitor  to  determine  range.  From  printed  materials 
provided  by  U-COFT,  it  is  not  possible  to  unequivocally  evaluate  gunner's 
tracking  skill.  On  the  other  hand,  it  is  not  particularly  difficult  for 
the  U-COFT  I/O  to  monitor  range  or  to  observe  the  gunner's  sight  picture 
during  engagements.  Thus,  while  U-COFT  provides  useful  information,  it  is 
not  complete  in  terms  of  the  detailed  gunnery  domain  analysis  they  we  have 
set  forth.  Complete  diagnostic  testing  would  require  additional  record 
keeping  by  the  U-COFT  I/O. 


Table  6-4 


U-COFT  Performance  Feedback 


Skill  Dimensions 


Criteria 


Target  Acquisition 


Reticle  Area 


System  Management 


Time  to  acquire  target 

Number  of  Identification  and  Classification 
Errors 

Time  to  fire  first  round  or  burst 
Time  to  kill 

Magniture  of  main  gun  aiming  error 

Number  of  switch  setting  errors  before  firing 
Number  of  switch  setting  errors  at  the  time  of 
iring. 


aSk i 1 1  dimension  score  is  the  score  of  the  lowest  criterion  within  the 
dimension.  Possible  scores  are  A,  B,  C  or  F. 

^Each  criterion  appears  on  the  Performance  Analysis  printout  in  raw  form 
(e.g.,  time,  number  of  errors). 


Although  we  suggest  that  U-COFT  is  useful  for  testing  a  large  portion 
of  the  tactical  gunnery  domain,  there  is  not  sufficient  information  to 
indicate  how  U-COFT  performance  can  be  translated  into  performance 
proficiency  on  the  real  equipment.  There  are  several  reasons.  First, 
U-COFT  performance  level,  in  terms  of  a  crew's  position  in  the  matrix 
progression,  is  a  function  of  how  much  the  crew  has  practiced  on  U-COFT  as 
well  as  its  skill.  Highly  skilled  crews  just  beginning  U-COFT  training 
will  be  low  in  the  matrix  because  they  have  not  had  time  to  move  up. 
Second,  SMEs  using  U-COFT  have  reported  some  dissatisfaction  with  the 
matrix;  the  order  of  difficulty  of  exercises  within  the  of  the  matrix  has 
recently  been  adjusted.  Third,  there  is  no  good  external  criterion  of 
tactical  gunnery  to  use  as  a  reference.  Table  VIII  measures  marksmanship. 
On  the  other  hand,  U-COFT  trains  primarily  marksmanship,  so  there  ought  be 
a  relationship  between  U-COFT  and  Table  VIII  scores.  At  least  two  studies 
have  examined  U-COFT  practice  and  Table  VIII  scores  (Rapkoch  &  Robinson, 
1986;  and  Hughes,  Butler,  Sterling,  &  Berglund,  1987).  Only  the  later 
study  looked  at  correlations  across  crews  between  amount  and  terminal 
level  of  U-COFT  practice  and  Table  VIII  indices.  Correlations  were  too 
low  to  support  a  conclusion  that  the  U-COFT  data  and  the  Table  VIII  data 
were  measuring  the  same  skills. 
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VIGS  and  TopGun.  Both  of  these  devices  gives  "points"  based  on  hits 
and  speed.  For  either  device,  it  is  not  possible  to  determine  the 
comparability  of  the  points  across  engagements  without  empirical  data. 
Likewise,  it  is  not  possible  to  project  any  device  point  scores  into 
expected  performance  on  real  equipment. 


Adaptive  Testing 


None  of  the  devices  were  designed  for  adaptive  testing.  VIGS  and 
TopGun  are  programmable  in  that  a  sequence  of  engagements  can  be  selected 
and  then  presented.  Selecting  exercises  one  at  a  time  is  time-consuming 
enough  that  there  is  no  gain  over  presenting  a  series  of  engagements  of 
varying  difficulties.  U-COFT,  with  its  large  array  of  exercises,  may 
appear  suited  to  adaptive  testing.  However,  since  U-COFT  engagements  are 
"packaged"  in  exercises  of  5  to  10  engagements  each,  selecting  engagements 
means  selecting  exercises  that  must  then  be  loaded  into  the  U-COFT 
computer.  Loading,  then  printing  results  and  unloading  exercises  take 
several  minutes.  Graham  (1986)  reported  that  execution  of  a  special  32 
engagement  test  using  portions  of  preselected  exercises  was  "awkward" 

(p.  5)  and  resulted  in  some  lost  data.  Adding  branching  decisions  for 
adaptive  testing  to  such  procedures  would  only  confuse  the  process 
further.  In  summary,  adaptive  testing  does  not  appear  practical  for  any 
of  the  devices. 


Conclusions 


As  simulators,  the  devices  reviewed  would  be  expected  to  have  varying 
levels  of  fidelity  and  inclusiveness.  Furthermore,  each  device  may  be 
expected  to  have  some  unique  operating  requirements.  For  training,  this 
means  that  personnel  may  practice  on  the  devices  but  they  must  also 
practice  on  the  real  equipment  to  make  the  adjustment  for  the  differences 
between  the  two.  The  implications  of  testing  are  not  as  simple.  A 
vehicle  operation  analogy  may  illustrate.  Most  of  the  skills  of  driving  a 
car  are  the  same  whether  the  vehicle  is  a  automatic  or  standard 
transmission.  However,  if  a  person  who  has  learned  to  drive  only  cars 
with  automatic  transmissions  were  to  take  a  driving  test  in  a  standard 
transmission  vehicle,  failing  the  test  would  not  be  surprising. 

Similarly,  if  a  person  who  has  driven  only  standard  transmission  cars 
drives  one  with  an  automatic  transmission,  the  likelihood  of  attempting  to 
clutch  and  accidentally  hitting  the  brake  pedal  while  travel  20  to  30  mph 
may  be  reasonable  high.  Unfamiliarity  with  one  type  of  vehicle  or  another 
reduces  the  expected  relationship  of  performance  between  the  two  types. 

On  the  other  hand,  for  persons  who  regularly  drive  both  types,  the 
correlation  of  driving  test  scores  between  the  two  types  would  be  expected 
to  be  quite  high. 
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A  similar  problem  occurs  with  correlation  of  simulator  performance 
with  real  equipment  performance.  The  correlation  would  be  expected  to  be 
higher  for  person  who  regularly  practice  on  both  than  for  persons  who 
practice  on  one  but  not  the  other.  Thus,  our  evaluations  emphasize 
fidelity  (i.e.  similarity  between  device  and  tank)  and  are  more 
conservative  for  testing  than  for  training. 


CHAPTER  7 


DEVICE-BASED  TRAINING  AND  TESTING  STRATEGY 


The  present  chapter  combines  information  from  the  previous  two 
chapters  by  assigning  training/testing  objectives  and  devices  to  units  of 
instruction.  The  outcome  of  this  process  is  a  proposed  strategy  for 
training  gunnery  skills  using  the  designated  training  devices.  In 
previous  chapters  we  have  advocated  the  interspersing  of  device-based  and 
on-tank  training.  That  position  is  maintained.  For  simplicity,  the 
proposed  program  presents  only  the  device-based  portion  of  training.  We 
assume  that  the  Ml  tank  can  and  should  be  used  for  training  and  practice 
in  all  aspects  of  gunnery. 

It  should  be  cautioned  at  the  outset  that  the  training  strategy 
outlined  in  the  present  chapter  should  be  regarded  as  only  an  initial 
concept  since  the  training  and  testing  objectives  have  not  been  confirmed 
by  external  subject  matter  experts.  Should  the  objectives  be  modified,  it 
is  likely  that  the  overall  training  strategy  will  change.  The  purpose  of 
this  proposed  concept  is  to  address  some  of  the  training  design  issues 
early  in  program  development,  and  to  provide  a  "strawman"  for  reaction 
from  external  sources. 

Previous  analyses  were  algorithmic  in  that  they  followed  specific 
procedures.  In  contrast,  the  process  of  assigning  objectives  and  devices 
to  blocks  of  instruction  was  more  heuristic  because  it  was  constrained  by 
some  general  rules  of  thumb  rather  than  specified  by  procedural  steps. 
These  general  rules  may  be  summarized  as  follows: 

1.  Specify  a  generic  program.  The  proposed  training  strategy  was 
designed  to  serve  as  a  model  for  developing  any  gunnery  training  program. 
Consequently,  it  was  not  tied  to  any  particular  Armor  School  program  of 
instruction  (POI).  That  is,  the  program  was  designed  without  any 
assumptions  about  who  is  being  trained  or  how  much  training  time  is 
available. 

2.  Focus  on  the  designated  devices.  Although  there  are  numerous 
devices  for  training  gunnery  skills,  the  training  strategy  was  contrived 
to  maximize  use  of  the  four  computer-based  devices  that  were  designated 
for  study  in  the  present  project.  Additional  devices  or  media  were 
specified  only  when  necessary  to  provide  prerequisites  for  training  on  the 
devices  or  where  the  designated  devices  could  not  provide  the  training  on 
aspects  of  the  domain  itself. 

3.  Organize  objectives  into  meaningful  units  of  instruction.  The 
units  of  instruction  should  be  should  be  congruent  with  although  not 
necessarily  isomorphic  with  the  partitioning  of  armor  act.vities 
(Chapter  2).  In  other  words,  there  does  not  have  to  be  a  unit  of 
instruction  for  every  activity;  nevertheless,  instructional  units  ought  to 
allow  practice  of  meaningful  units  of  behavior  as  identified  in  the 
analysis  of  behaviors. 


4.  Specify  sequential  dependencies  between  instructional  units.  The 
instructional  units  will  be  sequenced  following  the  prerequisite 
relationships  identified  in  the  hierarchical  analysis.  This  information 
will  be  supplemented  wren  the  successive  elaboration  strategy  described  in 
Chapter  4. 

5.  Be  mindful  of  realistic  constraints.  Despite  its  experimental 
nature,  the  proposed  device-based  training  program  should  be  constrained 
by  realistic  training  considerations.  A  basic  constraint  is  that  students 
and  instructors  cannot  move  at  will  between  devices  within  a  single  unit 
of  instruction.  Therefore,  the  units  specified  a  single  device  for 
instruction  of  skills.  Furthermore,  the  order  in  which  instructional 
units  are  completed  should  be  mindful  of  real-world  constraints,  such  as 
the  availability  of  devices  and  other  equipment.  Meeting  this  constraint 
was  facilitated  by  the  similarity  in  results  between  the  training  and 
testing  capabilities  evaluations. 

6.  Prescribe  an  iterative  training  strategy.  There  is  considerable 
overlap  in  what  gunnery  devices  train.  The  training  strategy  should 
exploit  this  characteristic  to  provide  multiple  experiences  on  gunnery 
training  objectives  in  an  increasingly  realistic  context. 

The  next  two  sections  describe  the  process  as  well  as  the  outcome  of 
assigning  (a)  objectives  to  units  of  instruction  and  (b)  devices  to  units 
of  instruction.  Although  described  separately  below,  the  two  processes 
actually  occurred  in  parallel  in  order  to  best  meet  the  criteria  described 
above.  The  following  section  provides  a  discussion  of  training  devices 
for  proficiency  measurement  outside  of  the  training  context. 

ianment  of  Objectives  to  Units  of  Instruction 

The  hierarchical  analysis  of  gunnery  skills  described  in  Chapter  4 
revealed  a  total  of  235  training/testing  objectives.  Of  that  total,  91 
objectives  were  duplicates  of  objectives  presented  elsewhere  in  the 
domain.  Eliminating  these  duplicates  resulted  in  144  unique  objectives. 
Staying  mindful  of  the  training  objectives  and  the  devices  designated  for 
training,  we  laid  out  an  initial  hierarchy  of  instruction  units  along  with 
a  recommended  training  medium  or  device  for  each  unit  (see  next  section). 
Next,  the  144  training/testing  objectives  were  assigned  to  the  units.  On 
the  basis  of  this  first  attempt,  units  having  too  few  objectives  were 
subsumed  under  other  topics  and  units  having  too  many  objectives  were 
subdivided.  Objectives  were  reassigned  and  further  refinements  to  the 
hierarchy  were  made  until  the  training  strategy  appeared  both  effective 
and  practicable  according  to  guidelines  set  forth  in  the  previous  section. 
The  resulting  hierarchy  of  instructional  units  is  shown  in  Figure  7-1.  To 
further  describe  the  training  concept,  Table  7-1  presents  the  topic  of 
instruction  and  associated  media  or  devices  for  each  unit.  A  detailed 
list  of  objectives  for  each  unit  is  provided  in  Appendix  J. 


>  V  V  V  V 


Topics  and  Devices/Media  for  Instructional  Units 


Unit  Number  &  Title 

1. 

Conduct  of  Fire- 

--Basic  Skills  and  Knowledaes 

Topic: 

Basic  information  on  fire  commands  and  operation  of 
gunner  controls. 

Device/Medium: 

Classroom. 

2. 

Precision  Gunnery  I--Gunner  Duties 

Topic: 

Practice  on  basic  gunner  skills  (e.g.,  tracking  and 
lasing)  related  to  precision  engagements. 

Device/Medium: 

VIGS  or  TopGun. 

3. 

Secondary  Functions  I--PRE0PS  Procedures 

Topic: 

Instruction  and  practice  on  preparing  gunner,  loader, 
driver,  and  TC  stations  for  operation. 

Device/Medium: 

Tank. 

4. 

Secondary  Functions  1 1 — PREFIRE  Procedures 

Topic: 

Instruction  and  practice  on  preparing  the  tank  for 
combat. 

Device/Medium: 

Tank  in  Field  Context. 

5. 

Loader  Duties 

Topic: 

Instruction  and  practice  on  skills  related  to  loading 

Device/Medium: 

Tank . 

(table  continues) 


Unit  Number  &  Title 


6.  Driver  Duties 

Topic:  Instruction  and  practice  on  skills  related  to  driving. 

Device/Medium:  Tank. 

7.  Target  Acquisition/Ranqe  Determination 

Topic:  Instruction  and  practice  on  skills  related  to  target 

acquisition  and  range  determination. 

Pevice/Medium:  Field. 

8.  Precision  Gunnery  II--TC  Duties 

Topic:  Practice  on  basic  TC  skills  (e.g.,  gun  laying  and  fire 

commands)  related  to  precision  engagements. 

Device/Medium:  U-COFT. 

9.  Precision  Gunnery  1 1 1--TC/Gunner  Coordination 

Topic:  Practice  for  TC  and  Gunner  in  coordinating  skills 

related  to  precision  gunnery. 

Device/Medium:  U-COFT. 

10.  Coax  Engagements 

Topic:  Instruction  and  practice  on  skills  related  to  coaxial 

machine  gun  engagements. 

Medium/Device:  U-COFT  or  VIGS. 


(table  continues) 


Unit  Number  &  Title 


11.  Degraded  Mode  Gunnery  I--B/S,  GAS,  EMER.  &  MAN 

Topic:  Instruction  and  practice  on  when  and  how  to  use  the 

following  techniques  of  degraded  mode  gunnery: 
battlesight,  gunners  alternate  sight,  emergency  mode, 
and  manual  mode. 

Device/Medium:  U-COFT. 


12.  Degraded  Mode  Gunnery  II--$ystem  Failures 

Topic:  Instruction  on  what  to  do  in  case  of  failures  in  the 

following  systems:  laser  range  finder,  gunner's 
primary  sight,  thermal  imagery  sight,  crosswind 
sensor,  wind  sensor,  and  lead  angle  sensor. 

Device/Medium:  Classroom  supplemented  with  the  Hand-Held  Tutor. 


13.  Adjustment  of  Fire 

Topic:  Instruction  and  practice  on  techniques  for  adjusting 

fire. 

Medium/Device:  U-COFT. 


14.  Multiple  Returns 

Topic:  Practice  in  recognizing  and  reacting  to  multiple 

returns  from  the  laser  range  finder. 

Medium/Device:  U-COFT. 

15.  Assess  Results  of  Engagement 

Topic:  Instruction  on  crew  actions  performed  after  a  battle 

engagement. 

Medium/Device:  Classroom. 


(table  continues) 


Unit  Number  &  Title 


16.  Multiple  Tarqets--Basic  Knowledges 

Topic:  Instruction  on  fire  commands  and  crew  actions  related 

to  multiple  target  engagements. 

Medium/Device:  Classroom  supplemented  with  EIDS. 

17.  Simultaneous  Enqaqements--Practice 

Topic:  Practice  on  skills  related  to  simultaneous 

engagements . 

Medium/Device:  U-COFT. 

18.  Multiple  Enqaqements--Practice 

Topic:  Practice  on  skills  related  to  multiple  engagements. 

Medium/Device:  U-COFT. 

19.  Tactical  Gunnery  Exercise 

Topic:  An  integrative  gunnery  exercise  in  a  simulated 

tactical  context. 


Medium/Device:  SIMNET. 


As  can  be  seen  in  Figure  7-1,  the  instructional  units  are  numbered 
from  1  to  19  to  identify  units  as  well  as  to  provide  a  suggested  order  of 
presentation.  Three  aspects  of  the  course  sequence  should  be  elaborated 
upon.  First,  Unit  I  (Conduct  of  Fire)  provides  instruction  on  basic 
skills  and  knowledges  that  are  prerequisite  to  every  block  of  instruction. 
In  addition  to  the  specific  training  objectives,  this  unit  should  provide 
some  information  on  the  general  flow  of  activities  and  basic  terminology 
associated  with  gunnery.  The  purpose  of  this  information  is  to  provide 
"advance  organizers"  upon  which  the  student  may  conceptually  organize 
gunnery  procedures  (Ausubel,  1967).  Second,  the  sequencing  was  determined 
by  a  combination  of  an  successive  elaboration  strategy  (described  in 
Chapter  4)  and  the  prerequisite  relationships  between  units.  Note  that 
precision  gunnery  forms  the  "trunk"  from  which  most  of  the  elaborative 
branching  occurs.  Prerequisites  are  trained  just  prior  to  the  elaborative 
branching  to  which  it  pertains.  For  instance,  knowledge  prerequisites 
concerning  multiple  targets  (Unit  16)  are  trained  just  prior  to  training 
simultaneous  and  multiple  engagements  (Units  17  and  18).  Third,  practical 
constraints  were  also  considered  in  sequencing  units.  For  instance,  the 
four  units  for  which  the  tank  was  the  recommended  medium  were  sequenced 
together  to  minimize  the  logistics  required  in  obtaining  the  combat 
vehicle. 

As  indicated  in  Appendix  J,  certain  objectives  are  listed  in  more 
than  one  unit.  The  purpose  of  the  cross  listing  was  to  provide  iterative 
training  on  the  objectives  where  possible.  The  model  for  this  iterative 
training  strategy  is  the  crawl-walk-run  approach  cited  in  Army  training 
literature.  Although  typically  used  to  describe  three  progressive  phases 
of  training  collective  tasks,  Drucker  and  Morrison  (1987)  provided  general 
criteria  for  crawl,  walk,  and  run  phases  that  may  be  applied  to  individual 
tasks  as  well.  In  the  crawl  phase  of  training,  a  student  is  introduced  to 
a  particular  objective.  The  instructor  provides  verbal  information 
required  to  learn  the  objective  and  may  actually  demonstrate  task 
performance.  The  walk  phase  of  training  is  characterized  by  the  student's 
practicing  the  task  to  the  performance  standard.  However,  the  task  is 
practiced  in  isolation  from  other  related  tasks,  and  the  context  may  not 
be  realistic.  Finally,  the  run  stage  of  training  is  identified  by 
integrative  training  wherein  tasks  are  performed  with  other  related  tasks 
in  real-time  and  within  realistic  contexts.  Although  these  phases  of 
training  may  be  conceptually  distinct,  it  is  difficult  to  classify  actual 
training  activities  as  uniquely  typifying  one  or  the  other  phase  of 
training.  Consequently,  we  did  not  specify  particular  training  phases  for 
objectives  within  units.  Nevertheless,  the  proposed  training  strategy  is 
in  keeping  with  the  basic  notion  behind  the  crawl-walk-run  approach:  that 
objectives  be  iteratively  trained  in  increasingly  realistic  contexts. 


nment  of  Devices  to  Instructional  Units 


Training  devices  were  assigned  to  the  instructional  units  by 
considering  their  capabilities  as  outlined  in  Chapters  5  and  6.  T 
device/media  assignments  are  discussed  in  the  following  paragraphs 


U-COFT.  U-COFT  was  selected  most  often  (for  8  of  the  19  units)  as 
the  device  of  choice  for  the  proposed  program.  One  reason  it  was  chosen 
was  simply  because  some  activities  which  can  be  performed  on  U-COFT  cannot 
be  performed  on  any  other  device  (e.g.,  simultaneous  engagements).  In 
addition,  U-COFT  offers  the  highest  fidelity  and  greatest  flexibility  in 
terms  of  instructional  features.  As  stated  in  Chapter  5,  this  is  not  an 
unexpected  result  considering  the  cost  and  stated  purpose  of  U-COFT. 

VIGS/TopGun.  For  purposes  of  the  present  project,  VIGS  and  TopGun 
are  regarded  as  functionally  identical,  i.e.,  they  both  provide  practice 
on  many  of  the  same  prerequisite  skills.  Furthermore,  they  are  both 
portable  (relative  to  U-COFT  and  SIMNET),  and  allow  students  to  practice 
in  full  view  of  the  instructor  and  other  observers.  Thus,  either  device 
would  be  a  useful  classroom  adjunct.  In  the  proposed  program,  Precision 
Gunnery  I  (Unit  2)  immediately  follows  the  Conduct-of-Fire  class  to  allow 
students  to  apply  some  of  the  concepts  that  were  introduced  in  the  first 
class  as  well  as  practice  other  prerequisites  to  precision  gunnery.  In 
addition  to  Unit  2,  VIGS  may  also  be  useful  in  introducing  students  to 
coax  engagements  (Unit  10). 

SIMNET.  In  contrast  to  VIGS  and  TopGun,  SIMNET  is  designated  for  use 
only  in  the  advanced  stage  of  the  training  program.  As  revealed  in 
Chapter  5,  SIMNET  is  not  well  suited  to  initial  training  for  two  reasons: 

(a)  its  lack  of  training  features  makes  it  inappropriate  to  provide 
repetitive  training  necessary  in  the  early  stages  of  training;  and 

(b)  the  fidelity  of  driving  and  loading  activities  is  insufficient  to 
support  initial  training  on  these  skills.  On  the  other  hand,  SIMNET  is 
well-suited  to  provide  more  advanced  training  in  the  proposed  program  for 
another  pair  of  reasons:  (a)  it  is  the  only  device  to  simulate  many 
important  tactical  conditions;  and  (b)  it  is  the  only  device  that  supports 
full-crew  coordination.  Consequently,  SIMNET  was  designated  as  the  device 
to  provide  the  final  unit  of  instruction  consisting  of  an  integrative 
tactical  exercise. 

Adjunct  Devices.  Although  not  the  focus  of  the  present  study,  two 
devices  developed  by  ARI  and  the  Training  Technology  Agency  (TTA)  are 
designed  to  promote  the  acquisition  of  gunnery-related  knowledges.  These 
devices  are  designated  to  support  or  to  perhaps  substitute  for  classroom 
instruction  for  some  of  the  units.  These  adjunct  "knowledge"  devices  are 
briefly  described  below: 

1.  The  Hand-Held  Tutor  (HHT)  was  originally  developed  by  ARI  as  a 
small  and  inexpensive  electronic  device  for  providing  drill  and  practice 
on  technical  vocabulary.  ARI  and  the  Training  Technology  Field  Activity 
(TTFA)  have  since  modified  the  HHT  to  provide  training  on  single-  and 
multiple-target  fire  commands,  degraded  mode  gunnery,  and  multiple  return 
strategies. 

2.  ARI  and  TTFA  have  also  developed  courseware  for  the  TICCIT 
computer-based/videodisc  instructional  system.  Courseware  has  been 
developed  on  a  number  of  armor  subjects,  most  notably  single-  and 
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multiple-target  fire  commands.  Current  plans  are  to  convert  this  TICCIT 
courseware  to  the  Army-wide  system  for  computer-based/videodisc  instruc¬ 
tion,  the  Electronic  Information  Delivery  System  (EIDS). 

Alternative  Media.  Chapter  5  noted  a  number  of  fidelity  problems 
that  contraindicated  the  use  of  any  of  the  four  devices  for  training  some 
aspects  of  the  gunnery  domain.  The  following  alternative  media  were 
designated  for  those  problematic  aspects: 

1.  Tank.  The  tank  is  only  reasonable  medium  for  training  PREOPS  and 
PREFIRE  procedures,  loader  duties,  and  driver  duties.  Whereas  the  PREOPS 
procedures  may  be  trained  on  the  tank  in  a  motor  pool  setting,  PREFIRE 
procedures  may  require  access  to  a  range,  or  at  least,  a  field  setting. 

2.  Field.  Target  Acquisition/Range  Determination  (Unit  7)  and 
Prepare  for  Offense/Defense  (Unit  4)  should  be  trained  in  a  field  setting. 
However,  a  tank  may  not  be  necessary  to  train  these  objectives. 

3.  Classroom.  In  addition  to  the  initial  unit  on  Conduct  of  Fire, 
two  other  units  should  be  trained  in  a  classroom  setting: 

.  The  topics  covered  in  Unit  16  (Multiple  Targets--Knowledge)  may  be 
regarded  as  a  continuation  of  Unit  1  (Conduct  of  Fire)  in  that  it 
deals  primarily  with  fire  commands.  The  reason  for  splitting  this 
unit  from  Conduct  of  Fire  is  to  place  it  closer  to  the  point  in  which 
students  may  practice  the  multiple  target  skills  (Units  17  and  18). 

.  The  second  unit  on  degraded  mode  gunnery  (Specific  System  Failures) 
consist  of  degraded  modes  which  cannot  be  simulated  on  the  devices, 
e.g.,  lead  sensor  system  failure.  However,  the  skill  required  to 
respond  to  such  failures  is  minimal,  and  task  performance  is  more 
dependent  of  the  knowledge  related  to  detecting  the  failures 
themselves.  Therefore,  the  classroom  setting  may  be  sufficient  to 
learn  these  tasks. 


Proficiency  Testing  Outside  of  Training  Context 

The  above  specifications  focus  on  device  use  in  the  training 
context  where  evaluation  of  individual  or  crew  performance  is  considered 
an  integral  part  of  training.  Specification  of  particular  devices  for  the 
different  phases  of  training  therefore  implies  use  of  the  device  for 
testing  as  well  as  for  training.  On  the  other  hand,  proficiency  testing 
does  not  always  occur  in  the  training  context.  This  section  presents  some 
concerns  about  the  use  of  devices  to  assess  gunnery  skills  outside  of 
training. 

An  advantage  of  testing  in  the  training  context  is  that  the  skill  of 
those  being  tested  is  already  targeted  within  the  range  of  the  objective 
level  being  trained.  Outside  of  the  training  context,  there  may  be  less 
information  about  persons'  skill  levels  and  so  a  wider  range  of  skills  and 
objectives  must  be  presented  by  a  test.  U-COFT  is  the  only  device  that 
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covers  an  extensive  portion  of  the  range  of  skills.  This  makes  U-COFT  the 
obvious  choice  of  the  four  devices  for  determining  proficiency  levels  of 
gunners  and  tank  commanders.  On  the  other  hand,  if  VIGS  or  TopGun  is 
fielded  at  a  lower  organizational  level,  availability  may  make  them 
convenient  screening  devices  for  testing  gunners. 

There  are  three  additional  problems  that  will  require  further 
research  before  any  more  definitive  recommendations  can  be  made  for  using 
U-COFT,  or  any  of  the  other  device  for  testing.  First,  we  have  discussed 
the  problem  of  translating  device  scores  into  meaningful  "real  world" 
scores  (see  Chapter  6).  Even  if  the  two  problems  below  were  solved,  there 
would  need  to  be  additional  efforts  to  anchor  device  score'"  in  terms  of  an 
external  criterion  metric. 

Second,  the  extent  to  which  device  unique  performance  requirements 
disrupt  performance  of  those  without  device  experience  must  be  determined. 
As  noted  in  Chapters  5  and  6,  in  the  training  context  with  a  mix  of  device 
and  real  equipment  training,  personnel  can  learn  to  adjust  to  unique 
device  requirements.  On  the  other  hand,  without  such  alternating 
practice,  an  otherwise  competent  person  may  not  perform  well  on  an 
unfamiliar  device.  As  U-COFT  training  spreads  throughout  the  Army,  this 
will  become  less  of  a  problem.  However,  it  should  be  remembered  that  we 
are  being  speculative.  Extended  studies  of  U-C0FT/M1  performance 
relationships  are  still  needed. 

Third,  if  U-COFT,  or  another  device,  is  to  be  used  for  proficiency 
testing,  an  operational  test  must  be  developed.  Certainly,  the  U-COFT 
library  is  extensive  and  one  can  imagine  assembling  a  test  that  contains 
sample  engagements  from  a  range  of  activities  and  under  a  vary  of 
conditions.  However,  actually  assembling  a  test  is  not  as  easy.  Graham 
(1986)  did  string  together  pieces  of  several  engagements  for  research 
purposes  and,  even  in  those  controlled  conditions,  found  the  procedure  to 
be  difficult  to  implement.  Thus,  current  programming  of  U-COFT  training 
makes  efficient  testing  over  the  range  of  skills  difficult. 

In  summary,  our  current  conclusion  is  that  proficiency  testing  with 
any  of  the  four  devices  if  it  is  conducted  outside  of  the  training  context 
will  be  problematic.  Therefore  no  recommendations  are  being  made  for 
stand-alone  proficiency  testing. 


Conclusions 


The  concept  for  training  and  testing  indicates  that  it  is  feasible  to 
use  existing  computer-based  devices  to  train  and  test  much  of  the  domain 
of  tank  gunnery  in  an  integrated  fashion.  However,  there  are  gaps  in  the 
training  program  where  devices  do  not  support  substantial  portions  of  the 
domain.  For  instance,  none  of  the  devices  supported  training  on  target 
acquisition  or  degraded  mode  gunnery  for  certain  system  failures.  Also, 
there  were  cases  where  basic  skills  and  knowledges  were  not  integrated 
into  higher  level  training.  For  instance,  the  Units  3  and  4  (Secondary 
Functions)  are  not  integrated  into  any  later  instructional  units. 


Similarly,  the  skills  learned  in  Unit  17  (Simultaneous  Engagements)  are 
not  integrated  into  the  final  tactical  exercise,  simply  because  SIMNET 
does  not  support  Cal  .50  engagements.  Consequently,  on-tank  experience  at 
both  the  beginning  and  advanced  stages  of  training  is  necessary  to  train 
and/or  test  the  entire  domain  of  gunnery. 
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